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ABSTRACT 


The  Low  Level  Windshear  Alert  System  (LLWAS)  is  an  anemometer-based  surface  network 
used  for  detection  of  hazardous  wind  shear  and  acquisition  of  operational  wind  information  in  the 
airport  terminal  area.  The  quality  of  wind  data  provided  by  the  LLWAS  anemometers  is  important 
for  the  proper  performance  of  the  LLWAS  wind  shear  detection  algorithms.  This  report  describes 
the  development  of  an  automated  method  for  anemometer  data  quality  analysis  (DQA).  This  method 
identifies  potential  data  quality  problems  through  comparison  of  wind  data  from  each  sensor  within 
a  network  to  the  mean  wind  speed  and  direction  of  the  entire  network.  The  design  approach  and 
implementation  are  described,  and  results  from  testing  using  data  from  the  demonstration  Phase  III 
LLWAS  network  in  Orlando,  FL  are  reported.  Potential  improvements  to  the  automated  DQA 
algorithm  are  presented  based  on  experience  gained  during  analysis  of  the  Orlando  data.  These 
recommended  improvements  are  provided  to  assist  future  development  and  refinement  of  the  DQA 
methodology  to  be  performed  by  the  FA  A  Technical  Center. 
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1.  INTRODUCTION 


The  Low  Level  Windshear  Alert  System  (LLWAS)  is  currently  the  primary  mechanism  for 
detection  of  hazardous  wind  shear  and  acquisition  of  operational  wind  information  in  the  terminal 
area  at  more  than  100  U.S.  airports  [Goff  and  Gramzow,  1989;  Wilson  and  Gramzow,  1991].  The 
system  relies  cm  wind  measurements  taken  at  approximately  10-second  intervals  from  a  network 
of  anemometers  covering  the  airport  area.  LLWAS  algorithms  use  these  measurements  to  estimate 
local  divergence  (or  conveigence)  of  wind  that  is  evidence  of  potentially  hazardous  wind  shear.  This 
system  has  undergone  a  series  of  upgrades;  the  current  system  (Phase  II  LLWAS)  employs  a  network 
of  six  sensors  spaced  4  to  6  km  apart.  Plans  are  in  progress  for  deployment  of  a  more  sophisticated 
version  of  the  wind  shear  detection  algorithm  (Phase  III)  at  approximately  10  to  12  airports.  This 
upgrade  will  include  a  more  dense  network  of  sensors,  typically  consisting  of  12  to  20  anemometers 
spaced  approximately  2.5  km  apart. 

The  quality  of  wind  data  provided  by  the  LLWAS  anemometers  is  important  for  the  proper  per¬ 
formance  of  the  LLWAS  wind  shear  detection  algorithms.  Filters  in  the  algorithms  provide  protec¬ 
tion  against  occasional  spurious  wind  measurements.  However,  systematic  bias  or  error  in  the  wind 
measurements  can  cause  degraded  wind  shear  detection  performance  either  by  causing  missed  wind 
shear  detections  or  by  causing  false  alerts.  In  addition,  for  reasons  of  economy,  LLWAS  sensor  net¬ 
works  are  designed  to  provide  minimal  redundant  coverage  for  wind  shear  detection.  Thus,  high 
data  quality  from  each  sensor  in  the  network  is  required  for  proper  system  performance. 

Degradation  in  sensor  performance  resulting  in  unacceptable  data  quality  is  both  difficult  and 
costly  to  detect  through  manual  analysis  since  many  of  the  potential  problems  are  subtle  or  intermit¬ 
tent.  Because  of  the  importance  of  data  quality  and  the  large  number  of  deployed  sensors,  the  FAA 
is  interested  in  an  automated  method  for  data  quality  monitoring.  The  current  on-line  capability  al¬ 
lows  detection  of  obvious  hardware  or  communications  failures  but  is  unable  to  identify  more  subtle 
problems  or  a  gradual  degradation  in  performance.  MIT  Lincoln  Laboratory  has  been  working  to 
develop  an  automated  LLWAS  Data  Quality  Analysis  (DQA)  method  that  will  identify  these  types 
of  sensor  problems  in  a  timely  manner  so  as  to  avoid  degraded  LLWAS  wind  shear  detection  perfor¬ 
mance.  Primary  development  and  testing  of  a  viable  methodology  was  performed  using  anemome¬ 
ter  data  collected  as  part  of  Lincoln  Laboratory’s  Terminal  Doppler  Weather  Radar  (TDWR)  (Tum- 
bull  et  al.,  1989)  testbed  in  Orlando,  FL  during  1991  and  1992  and  using  data  collected  during  1 993 
as  part  of  the  Phase  III  LLWAS  Operational  Test  &  Evaluation  in  Orlando. 

The  objective  is  an  algorithm  that  automatically  runs  in  real  time  as  part  of  the  LLWAS  system. 
An  intermediate  step  may  be  to  process  data  off  line  using  the  automated  methodology  at  a  central 
facility.  The  FAA  requested  from  Lincoln  Laboratory  a  document  detailing  the  functional  require¬ 
ments  of  an  automated  analysis  system  so  that  it  could  be  included  as  part  of  the  system  requirements 
for  a  future  LLWAS  procurement.  Results  and  experience  from  the  off-line  DQA  testing  done  from 
1991  through  1993  were  used  to  develop  these  system  functional  requirements. 

This  report  documents  the  development  of  the  DQA  methodology  to  date  and  recommends  im¬ 
provements  for  future  development  and  implementation.  It  includes  the  rationale  for  selecting  a  de¬ 
sign  approach,  an  overview  of  the  design,  and  some  of  the  details  regarding  implementation  of  the 
algorithm.  It  also  includes  a  description  of  algorithm  performance  and  lessons  learned  during  off¬ 
line  testing  using  anemometer  data  from  the  Orlando  testbed.  Experiences  from  this  testing  indi- 
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cated  several  key  improvements  to  the  algorithm  that  would  make  it  a  more  reliable  automated  data 
analysis  method  These  improvements  are  described  and  were  taken  into  consideration  during  de¬ 
velopment  of  the  DQA  functional  requirements  document  for  the  FAA  (See  Appendix  A).  The  de¬ 
scription  of  algorithm  processing  and  experimental  results  presented  here  also  represent  a  reason¬ 
able  baseline  for  any  future  development  of  a  formal  algorithm  specification. 
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2.  DEVELOPING  AN  ANALYSIS  APPROACH 


2.1  Scientific  and  Engineering  Considerations 

The  purpose  of  making  wind  measurements  at  an  LLWAS  sensor  is  to  obtain  a  reliable  estimate 
of  the  wind  velocity  in  the  region  near  the  sensor,  which  represents  the  wind  10  m  above  a  fairly 
smooth  surface.  The  10  m  height  is  a  standard  established  by  the  FAA  for  winds  representative  of 
the  airport  area  (Wieringa,  1980).  For  purposes  of  this  discussion,  we  need  to  consider  measurement 
of  the  local  wind  under  two  separate  general  conditions:  steady-state  ambient  wind  flow  resulting 
from  larger-scale  geostrophic  forcing  and  mesoscale  winds  in  the  presence  of  local  low  level  wind 
shear.  Characteristic  of  the  former  condition  is  a  vertical  wind  profile  in  which  the  wind  speed  in¬ 
creases  gradually  with  height  for  several  hundred  meters  due  to  the  frictional  effect  near  the  earth  s 
surface.  Under  the  latter  conditions,  this  profile  is  disrupted  by  the  local  dynamics  of  the  wind  shear 
event  During  such  an  event,  wind  energy  is  transported  near  the  ground  by  strong  downdrafts  such 
that  the  vertical  wind  profile  typically  shows  the  strongest  winds  near  the  ground  (i.c.,  in  the  lowest 
100-200  meters  above  ground  level). 

We  wish  to  detect  errors  in  wind  measurements  by  monitoring  the  wind  measurements  during 
the  steady-state  wind  conditions,  during  which  time  the  network  of  LLWAS  sensors  represents  an 
over-sampling  of  the  wind  field  in  the  airport  area.  The  degree  to  which  a  sensor  measurement  fails 
to  be  regionally  representative  is  viewed  as  one  contribution  to  measurement  error.  The  sources  of 
these  errors  fall  into  one  of  two  broad  categories: 

a.  Inadequate  sensor  siting  or 

b.  Equipment  malfunction  (mechanical  or  electrical). 

Both  inadequate  sensor  siting  and  equipment  malfunctions  can  result  in  wind  measurement  error 
distributions  that  are  biased  in  speed  or  direction  and  that  have  excessive  variance. 

We  have  chosen  a  method  of  analysis  that  involves  the  comparison  of  the  winds  at  each  sensor 
with  the  winds  measured  over  the  entire  network.  The  patterns  of  differences  in  these  winas  is  the 
basis  for  the  LLWAS  wind  shear  detection  algorithm.  When  wind  shear  is  not  present  and  when  the 
wind  speed  exceeds  3  m/s,  there  is  significant  uniformity  in  the  observed  wind  field.  At  these  times 
there  is  a  significant  correlation  of  the  surface  winds  observed  at  proximate  locations.  The  cnor 
variance  increases  with  distance  until  local  topographic  effects  decorrelate  the  winds.  For  example, 
locations  separated  up  to  3  km  exhibit  a  variability  of  about  seven  percent  [Wieringa,  1980].  Our 
analysis  shows  that  over  the  time  of  a  few  days,  there  is  enough  consistency  of  the  winds  over  the 
14-16  sensor  LLWAS  networks  at  Denver  and  Orlando  so  that  these  comparisons  can  provide  the 
basis  for  a  reliable  analysis  of  sensor  performance.  The  same  may  not  be  true  for  larger  networks 
where  it  may  be  necessary  to  partition  the  network  into  a  few  clusters  of  sensors  and  to  apply  the 
analysis  separately  to  each  cluster. 

2.1.1  Inadequate  Sensor  Siting 

Proper  sensor  siting  depends  on  a  variety  of  factors  that  are  described  in  detail  in  (Simiu  and 
Scanlan,  1986]  and  FAA  Order 6560.21  A  [FAA,  1 989].  The  problem  is  that  the  sensor  can  correctly 
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measure  the  winds  at  its  location,  but  that  these  winds  have  been  so  modified  by  local  effects  that 
they  reflect  a  sub-scale  phenomena  and  are  not  representative  of  the  nearby  winds.  There  are  three 
primary  effects  that  can  negatively  influence  that  measurement  of  regionally  representative  winds 
at  a  sensor  position:  surface  roughness,  sheltering,  and  channeling. 

Wind  speed  is  reduced  near  a  rough  surface  due  to  friction  and  turbulence.  For  steady-state 
wind  conditions,  a  simple  mathematical  model  of  this  speed  reduction  near  the  surface  incorporates 
two  parameters,  the  roughness  exponent  a  and  the  effective  surface  height  d.  The  roughness  expo¬ 
nent  depends  on  the  surface  roughness  and  the  stability  of  the  boundary  layer.  Over  a  uniform  tree 
canopy,  the  effective  surface  height  is  below  but  near  the  tree  tops  rather  than  at  ground  level.  During 
a  wind  shift  or  wind  shear  event,  the  vertical  dynamics  of  the  situation  control  the  vertical  structure. 
Surface  effects  may  be  minimal  at  this  time  and  are  difficult  to  analyze  theoretically  or  by  modeling. 
A  steady-state  wind,  after  a  brief  period  of  time,  typically  1  to  2  minutes,  will  establish  a  vertical 
wind  speed  profile  which  varies  from  nearly  zero  at  the  effective  surface  to  its  free  flow  speed  at 
a  few  hundred  meters,  and  whose  increase  with  height  above  the  effective  surface  is  reasonably  de¬ 
scribed  by 


u(z)«  u(r)  [  (z-d)  /  (r-d)  ]a 


(Eq.2-1) 


where  z  is  the  height  above  the  surface,  r  is  the  reference  height  and  u(r)  is  the  wind  speed  at  the 
reference  height.  For  a  neutral  boundary  layer,  a  varies  in  value  from  0.10  over  flat  terrain  to  0.25 
over  nonuniform  forests  and  irregularly  developed  urban  areas.  Values  up  to  0.6  have  been  observed 
in  stable  boundary  layers.  An  important  observation  is  that  any  error  from  this  effect  is  proportional 
to  the  wind  speed. 

The  vertical  wind  speed  profile  at  a  senses'  position  may  vary,  depending  on  the  direction  of  the 
wind.  A  typical  situation  at  an  airport  is  that  the  area  near  the  runways  is  cleared  and  the  area  outside 
the  airport  may  be  forested  or  developed.  The  result  is  that  both  the  effective  surface  height  and  the 
up-wind  roughness  exponent  may  depend  on  the  wind  direction  [Wieringa,  1980].  In  most  situa¬ 
tions,  this  effect  will  result  in  the  sensor  winds  having  a  slight  high  bias  from  some  directions  and 
a  slight  low  bias  from  others.  If  the  discrepancy  is  too  great,  the  only  solution  is  to  relocate  the  sensor 
site  [FAA,  1989].  Since  the  maximum  LLWAS  pole  height  is  50  m,  the  high  bias  is  usually  only 
a  few  percent,  although  biases  up  to  15  percent  are  possible.  Over  a  tall  forest,  the  effective  surface 
may  be  25  m  above  the  ground  and  it  is  difficult  to  avoid  a  low  bias  of  20  percent  or  more.  In  cases 
where  installation  restrictions  require  that  an  exceptional  bias  be  accepted,  then  the  sensor  perfor¬ 
mance  evaluation  algorithm  should  make  use  of  this  information  to  avoid  issuing  unwarranted 
trouble  alerts. 

One  may  be  tempted  to  use  this  knowledge  to  tune  the  data  from  each  sensor  to  eliminate  the 
bias  [Wieringa,  1976].  This  approach  may  have  value  for  operational  wind  information  (i.c.,  during 
steady-state  wind  conditions),  but  caution  is  advised  in  the  case  of  wind  measurements  for  wind 
shear  algorithms.  The  vertical  wind  structure  may  take  a  couple  of  minutes  to  re-establish  after  the 
completion  of  a  wind  shift  cm- wind  shear  event,  and  the  compensation  would  be  erroneous  at  these 
times,  precisely  the  time  when  accurate  winds  are  desired.  Making  such  compensations  has  the  po¬ 
tential  both  to  reduce  detections  and  to  induce  false  alerts.  This  is  not  a  situation  where  current  scicn- 


4 


tific  understanding  can  readily  provide  guidance.  Therefore,  it  would  be  unwise  to  make  such 
corrections  when  the  winds  are  to  be  used  for  wind  shear  computations. 

Sheltering  and  channelling  refer  to  the  respective  alteration  of  wind  speed  and  direction  as  the 
winds  flow  around  obstacles.  At  positions  close  to  the  obstacle,  both  severe  sheltering  and  channel¬ 
ling  are  possible.  An  anemometer  should  either  be  placed  sufficiently  high  above  the  obstacle  or 
at  a  location  sufficiently  far  away  to  avoid  this  situation  [FAA,  1989].  At  greater  distances,  channel¬ 
ling  is  not  likely,  but  some  modest  speed  reduction  is  still  a  possibility.  In  this  case,  the  reduction 
is  by  a  nearly  fixed  percentage  of  wind  speed.  The  LLWAS  Siting  Order  provides  guidelines  for 
locating  sensors  so  that  they  are  not  subject  to  channelling  and  they  incur  no  more  than  20  percent 
low  speed  bias  due  to  surface  roughness  and  sheltering. 

2.1.2  Equipment  Malfunction 

Any  attempt  to  infer  the  nature  of  an  equipment  failure,  from  symptoms  exposed  by  data  quality 
analysis,  must  involve  some  consideration  of  the  design  of  the  sensing  system.  We  have  applied 
these  techniques  to  data  from  the  LLWAS  installations  by  Loral  Data  Systems  at  the  Orlando  and 
Denver  airports.  These  systems  use  prop  and  vane  anemometers.  We  have  also  evaluated  data  from 
the  MIT  Lincoln  Laboratory  mesonet,  which  uses  cup  and  vane  anemometers.  This  equipment  is 
similar  to  the  Climatronics  LLWAS  13  installations.  While  there  are  some  differences  in  the  data  that 
results  from  these  two  systems,  the  dominant  factor  is  that  both  systems  measure  speed  and  direction 
directly.  Therefore,  to  understand  the  nature  of  an  equipment  failure,  it  is  necessary  to  apply  tests 
that  separately  analyze  speed  error  and  direction  error.  Other  anemometer  designs  involve  measur¬ 
ing  two  or  more  directional  components  of  the  horizontal  wind  and  mathematically  computing  speed 
and  direction.  For  these  sensors,  the  addition  of  specific  analysis  of  the  errors  in  the  measured  com¬ 
ponents  may  be  useful  for  detecting  certain  sensor  failures. 

2.2  Design  Overview 

The  LLWAS  Data  Quality  Analysis  (DQA)  algorithm  and  software  was  designed  to  work  in 
conjunction  with  the  LLWAS  Wind  Shear/Microburst  (WSMB)  detection  algorithms.  As  such,  it  is 
able  to  take  advantage  of  the  LLWAS  data  acquisition  function,  network  configuration  parameters, 
algorithm  control  parameters,  and  wind  shear  alert  information.  The  relationship  between  LLWAS 
and  DQA  processing  is  illustrated  in  Figure  2-1. 

Since  most  sensor  problems  produce  characteristic  effects  in  the  wind  speed  and/or  direction 
measurements,  the  general  approach  of  the  DQA  is  to  continually  compare  the  wind  speed  and  direc¬ 
tion  from  each  sensor  within  the  network  to  some  standard  wind  speed  and  direction.  The  standard 
for  comparison  was  chosen  as  the  mean  values  of  wind  speed  and  direction  of  all  sensors  within  the 
network,  in  the  absence  of  wind  shear.  The  assumption  here  is  that  the  network  mean  wind  is  a  fair 
representation  of  the  the  true  local  wind,  and  a  consistent  departure  from  the  mean  by  an  individual 
sensor  (when  no  wind  shear  is  present)  is  indicative  of  a  sensor  problem.  In  addition,  the  characteris¬ 
tics  of  the  departure  are  often  an  indicator  of  the  nature  of  the  problem. 

In  order  to  choose  a  method  for  comparing  individual  sensor  measurements  to  the  mean  wind, 
we  consider  the  expected  effects  of  a  malfunctioning  sensor  on  the  measurement  characteristics. 
Most  speed  sensing  problems,  such  as  bearing  drag  and  sheltering,  reduce  speed  measurements  pro¬ 
portionately  rather  than  by  some  discrete  increment.  For  this  reason,  we  choose  to  examine  speed 
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Figure  2-1.  Functional  relationship  between  LLWAS  wind  shear  detection  processing  (top  row)  end  data 
qualm  analysis  (DQA)  processing  (bottom  row). 


measurements  as  the  ratio  of  an  individual  sensor’s  value  to  the  vaiue  of  the  network  mean  speed. 
In  contrast  to  speed  errors,  wind  direction  problems  are  best  characterized  by  a  discrete  difference 
(including  sign)  of  the  direction  measurement  from  the  network  mean  direction. 

The  basic  algorithm  processing  is  as  follows:  for  each  poll  of  LLWAS  data  (approximately  once 
every  ten  seconds),  the  ratioof  wind  speed  at  each  sensor  and  the  network  mean  wind  speed  is  com¬ 
puted;  similarly,  the  difference  between  each  sensor’s  wind  direction  and  the  network  mean  direc¬ 
tion  is  computed.  Using  these  ratios  arid  differences,  a  statistical  data  base  is  continually  updated 
that  reflects  these  speed  and  direction  characteristics  for  each  station.  For  wind  speed  ratio,  data  are 
maintained  separately  for  various  ranges  of  mean  wind  direction  since  speed  sensing  problems  may 
be  directionally  dependent,  e.g.,  due  to  sheltering  from  a  nearby  obstruction.  Once  data  from  a  suffi¬ 
cient  number  of  polls  have  been  accumulated,  they  are  used  tocreate  percentage  frequency  distribu¬ 
tions  that  indicate  the  characteristics  of  the  speed-ratio  and  direction-difference  profiles  of  the  wind 
data  from  each  sensor.  These  distributions  are  then  compared  to  thresholds  derived  from  "expected” 
distributions  of  speed  ratio  and  direction  difference,  based  on  very  large  samples  of  data  from  all 
of  the  sensors  within  the  network.  A  significant  deviation  from  the  expected  distribution  indicates 
a  potential  problem  with  that  sensor.  Table  2-1  illustrates  the  relationship  between  typical  data  ab¬ 
normalities  and  the  likely  associated  sensor  problem. 
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TABLE  2-1 

Ust  of  Potential  Symptoms  Indicated  by  Wind  Speed  and  Direction  Statistics  and  the 
Corresponding  Problems  with  Which  They  Are  Likely  Associated 


SYMPTOM 

PROBLEM 

all  speed  ratios  near  zero 

catastrophic  e'ectro-mechanical  failure 

low  speed  bias,  directionally  dependent 

sheltering 

low  speed  bias,  directionally  independent 

frictional  drag  on  speed  sensor,  siting  problem  (low 
sensor  height) 

high  speeds  bias 

siting  problem  (wind  channeling  or  sensor  height) 

direction  offset 

misorientation,  loose  direction  mounting,  sticky 
direction  bearings 

flat  distribution  of  direction  differences,  without 
direction  offset 

loose  direction  mounting,  sticky  direction  bearings 
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3.  DQA  ALGORITHM  IMPLEMENTATION 


This  section  document  provides  a  more  specific  description  of  the  functional  concepts  and  de¬ 
sign  implementation  associated  with  the  DQA  algorithm.  Descriptions  arc  broken  down  into  four 
general  areas: 

a.  Data  preparation  /  preprocessing 

b.  Accumulating  wind  speed  and  direction  counts 

c.  Preparing  accumulated  data  for  testing 

d.  Performing  tests 

These  descriptions  reflect  design  implementation  employed  during  1992  testing  using  data 
from  the  Orlando  testbed.  A  discussion  of  improvements  to  this  version  is  presented  later  on  in  this 
report. 

3.1  Data  Preparation  /  Preprocessing 


34.1  Input  Winds 

In  order  to  monitor  sensor  performance,  we  choose  to  process  and  analyze  the  raw  winds  (i.e., 
wind  values  as  measured  by  the  LLWAS  sensors)  rather  than  a  representation  of  the  wind  resulting 
front  any  data  preparation  or  pre-processing  performed  by  the  LLWAS  algorithm.  Within  the  DQA 
processing,  the  u.v  wind  components  of  the  raw  wind  from  each  sensor  are  convened  to  a  wind  speed 
(m/s)  and  wind  direction  (degrees  from  which  the  wind  is  blowing,  measured  clockwise  from  mag¬ 
netic  north). 

3.1.2  Counting  Polls  and  Accumulating  Data 

The  DQA  algorithm  operates  by  periodically  performing  threshold  tests  on  accumulated  statis¬ 
tical  data.  The  frequency  with  which  the  various  tests  are  performed  is  keyed  to  the  number  of  total 
polls  of  LLWAS  data  processed.  Upon  initialization,  a  running  count  of  the  total  number  of  polls 
processed  is  maintained.  This  overall  poll  count  is  used  to  determine  the  frequency  with  which  to 
perform  various  tests  for  updating  the  status  of  each  sensor.  During  1992  testing,  this  frequency  was 
set  to  perform  testing  on  each  24-hour  period  of  data. 

3.1.3  No  Wind  Shear  Requirement 

An  underlying  assumption  to  the  approach  presented  here  is  that  the  wind  speed  and  direction 
from  a  properly  functioning  sensor  will  not  deviate  substantially  from  the  network  mean  wind,  as 
we  have  described  the  anemometer  network  as  an  oversampling  of  a  nearly  uniform  wind  field.  The 
exception  to  this  would  be  instances  where  there  is  a  shear  in  the  horizontal  wind  field  within  the 
sensor  network.  Under  this  circumstance,  one  would  expect  one  or  more  sensors  to  deviate  signifi¬ 
cantly  from  the  network  mean  wind.  Consequently,  the  use  of  data  for  compiling  sensor  statistics 
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relative  to  the  mean  wind  could  incorrectly  imply  a  sensor  problem.  In  order  to  avoid  this  scenario, 
compiling  of  statistical  data  for  all  sensors  is  suspended  during  the  presence  of  wind  shear  over  the 
network. 

It  should  be  noted  that  the  impact  of  this  effect  was  found  to  be  small  during  testing.  Essentially, 
it  is  significant  in  instances  when  the  only  “windy"  portion  of  the  overall  data  sample  occurs  during 
periods  of  wind  shear.  For  an  on-line  system,  however,  it  is  recommended  that  this  approach  be 
implemented  in  order  to  minimize  false  indications  of  a  sensor  failure  that  would  require  an  unneces¬ 
sary  response  from  maintenance  personnel. 

3.2  Comparison  with  Mean  Wind  Speed  and  Direction 

3.2.1  Computation  of  Standard  Wind 

The  basic  scheme  of  the  algorithm  is  to  continually  compare  the  wind  from  each  sensor  with 
some  estimate  of  some  ‘‘standard”  wind  speed  and  direction  in  order  to  detect  a  significant  departure 
as  evidence  of  a  sensor  malfunction.  Th  is  standard  wind  is  defi rned  as  the  mean  wind  speed  and  direc¬ 
tion  of  all  the  active  sensors  within  the  network.  The  use  of  the  network  mean  wind  as  the  standard 
implies  that  it  is  a  fair  representation  of  the  “true”  wind,  i.e.,  most  of  the  sensors  have  been  properly 
sited  and  a  majority  of  the  network  sensors  are  generally  assumed  to  be  in  good  working  order. 

The  standard  wind  speed  is  computed  each  poll  using  the  raw  data  from  each  sensor.  The  u,v 
components  from  each  station  arc  converted  toa  mean  wind  speed  and  direction.  Then  a  simple  arith¬ 
metic  mean  of  the  wind  speed  is  computed.  The  mean  direction  in  degrees  is  computed  from  the 
mean  u,v  values. 

3.2.2  Determination  of  Mean  Wind  Direction  Bin 

Since  deficiencies  associated  with  wind  speed  sensing  may  be  directionally  dependent,  e.g., 
sheltering  by  a  nearby  obstruction,  it  is  necessary  to  determine  wind  speed  characteristics  separately 
for  differing  wind  directions.  This  is  done  by  using  the  mean  wind  direction  as  a  directional  stan¬ 
dard.  Wind  speed  statistics  are  then  compiled  separately  for  various  “bins”  of  mean  wind  direction. 
This  is  done  by  identifying  ranges  of  mean  wind  direction  that  define  a  wind  direction  bin.  For  data 
processed  during  1992  and  1993,  the  direction  bins  were  defined  to  have  a  width  of  30  degrees.  For 
example,  the  first  direction  bin  was  centered  on  30  degrees  and  included  winds  from  15  degrees  to 
45  degrees;  the  second  bin  was  centered  on  60  degrees  and  included  winds  from  45  degrees  to  75 
degrees,  etc.  Thus,  for  each  new  poll  of  LLWAS  data,  speed  statistics  for  each  sensor  are  updated 
specifically  for  the  bin  corresponding  to  the  current  mean  wind  direction  bins.  An  additional  bin  in¬ 
cluding  wind  from  any  direction  is  also  updated  each  poll. 

3.2.3  Compiling  Speed  Ratio  Statistics 

For  each  poll  of  data  (approximately  once  every  10  seconds),  the  ratio  of  the  wind  speed  at  each 
sensor  with  the  mean  wind  speed  is  computed.  In  order  to  ensure  a  reliable  representation  (i.e.,  un¬ 
contaminated  by  the  effects  of  a  light  and  variable  wind),  ratios  arc  only  computed  for  polls  of  data 
for  which  the  mean  wind  speed  is  at  least  a  minimum  speed  (3  m/s). 
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The  speed  ratio  for  each  sensor  is  then  used  to  update  the  corresponding  speed-ratio  bin  count 
for  the  appropriate  mean  wind  direction  bin,  i.c.,  the  direction  bin  corresponding  to  the  mean  wind 
direction  during  the  current  poll.  The  speed  ratio  bins,  analogous  to  the  mean  wind  direction  bins, 
represent  a  range  of  speed  ratio  values.  For  instance,  the  “middle’’  speed  ratio  bin  includes  counts 
of  the  number  of  data  polls  for  which  an  individual  sensor  had  a  speed  ratio  whose  value  was  between 
0.90  and  1 . 10.  Thus,  after  processing  a  l?rge  number  of  data  polls,  the  result  was  a  matrix  of  counters 
that  indicated  the  frequency  distribution  of  counts  in  each  speed  ratio  bin,  sorted  by  mean  direction 
bin: 


Speed  Ratio  Bin 

0.00  to  0.50 
0.50  to  0.70 
0.70  to  0.90 
0.90  to  1.10 
1.10  to  1.30 
1.30  to  1.50 
1.50  to  2.00 
2.00  + 


Mean  Wind  Direction  Bin 

015-045  045-075  075-105  105-135  135-165 
X  X  X  X  X 

X  X  X  X  X 

X  X  X  X  X 

X  X  Speed  Ratio  Counters  X 

X  X  X  X  X 

X  X  X  X  X 

X  X  X  X  X 

X  X  X  X  X 


165-195... 

X 

X 

X 

X 

X 

X 

X 

X 


3.2.4  Compiling  Direction  Difference  Statistics 

For  each  poll  that  the  network  mean  wind  speed  is  at  least  a  minimum  speed  (3  m/s),  wind  direc¬ 
tion  differences  are  computed  for  each  sensor  whose  wind  speed  is  also  at  least  the  minimum  speed. 
(The  minimum  speed  requirement  is  used  to  disregard  large  direction  fluctuations  that  commonly 
occur  during  light  wind  conditions.)  Wind  direction  differences  are  computed  as  the  difference  be¬ 
tween  the  sensor  wird  direction  and  the  mean  wind  direction.  Direction  difference  is  represented 
in  degrees,  with  a  negative  value  indicating  a  direction  displacement  oriented  counter-clockwise 
from  the  network  mean  wind  direction. 

Analogous  to  speed  ratio,  direction  difference  bins  (with  each  bin  representing  some  range  of 
direction  differences)  are  used  to  update  wind  direction  counts  for  each  station.  Unlike  speed  ratio, 
however,  it  is  not  necessary  to  maintain  direction  difference  counts  foreach  network  mean  direction¬ 
al  bin.  For  each  computed  direction  difference,  the  counter  corresponding  to  the  proper  direction 
difference  bin  is  updated  separately  foreach  sensor.  The  direction  difference  bins  arc  defined  to  have 
a  width  of  10  degrees,  e.g.,  -95  to -85,  -85  to  -75, . . .,  -5  to  +5,  +5  to  +15, . . .,  +75  to  +85,  +85 
to  +95.  An  additional  “Extreme  High”  bin  is  maintained  to  count  all  direction  differences  of  greater 
than  95  degrees,  irrespective  of  sign. 

3.3  Performing  Tests 

Threshold  tests  using  the  statistical  wind  data  are  performed  periodically  at  an  interval  deter¬ 
mined  by  the  total  number  of  polls  elapsed  following  algorithm  initialization.  Prior  to  performing 
threshold  tests,  the  statistical  counts  compiled  for  speed  ratios  and  direction  differences  arc  con¬ 
vened  to  a  format  that  allows  testing  for  deviations  from  expected  values.  For  wind  speed,  this  in- 
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eludes  (for  each  sensor)  a  percentage  distribution  of  wind  speed  ratio  for  each  mean  wind  direction 
bin,  a  distribution  for  all  wind  directions  combined,  an  average  wind  speed  ratio  representing  each 
distribut.un,  and  the  number  of  valid  counts  used  togenerate  each  distribution.  An  example  of  speed 
ratio  statistics  used  for  testing  is  shown  in  Figure  3-1. 


STATION  «S 
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Figure  3-1.  Example  of  statistical  data  available  from  a  single  sensor,  used  for  wind  speed  threshold  testing. 
Included  are  station  number,  table  of  percentage  frequency  distributions  of  speed  ratio  sorted  by  mean  wind 
direction  bin.  average  speed  ratio  for  each  wind  direction,  and  number  of  valid  counts  comprising  the  distribu¬ 
tion  for  each  wind  direction  bin. 


Statistics  for  wind  direction  tests  include  a  percentage  distribution  of  wind  direction  difference . 
average  wind  direction  difference,  and  the  number  of  valid  counts  for  each  distribution.  An  example 
of  direction  difference  statistics  used  for  testing  is  shown  in  Figure  3-2. 

Once  the  counters  arc  converted  to  percentage  frequencies  and  average  values,  the  objective  is 
to  test  for  a  significant  deviation  from  expected  values  or  distribution  profiles.  The  only  discrete 
guideline  provided  in  FAA  standards  is  that  the  wind  speed  not  deviate  by  more  than  20  percent  from 
the  “true”  wind  speed,  where  the  true  speed  is  defined  as  that  measured  by  a  properly  functioning 
sensor  situated  10  meters  above  ground  level  and  totally  unsheltered  by  any  local  obstructions  such 
as  buildings,  trees,  topography,  etc. [FAA,  1989]  Although  this  is  a  necessary  criterion  for  testing, 
it  is  not  considered  sufficient  since  it  provides  no  standard  for  the  distribution  of  speed  measure¬ 
ments  and  does  not  address  direction  measurements  at  all. 

In  order  to  identify  instances  where  a  sensor  showed  a  significant  deviation  from  an  expected 
piofile  of  speed  ratio  or  direction  difference,  some  expected  profiles  were  established.  This  was  ini¬ 
tially  done  by  examining  a  very  large  sample  of  data  (hundreds  of  thousands  of  data  polls)  available 
from  the  prototype  Phase  III  LLWAS  network  deployed  at  Stapleton  International  Airport  in  Denver. 
Once  sufficient  data  was  available  for  processing,  similar  profiles  were  examined  for  Orlando,  and 
a  reasonable  binning  approach  was  established.  (The  bins  for  speed  ratio  and  direction  difference 
arc  adjustable,  but  the  values  chosen  for  implementation  are  shown  here.)  An  example  of  the  large- 
sample  percentage  frequency  distributions  of  speed  ratio  and  direction  difference  for  Orlando  arc 
shown  in  Figure  3-3  and  3-4,  respectively. 
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Figure  3-2.  Example  of  siarislical  data  available  for  wind  direction  threshold  testing.  Included  are  number  of 
total  elapsed  polls  following  initialization,  table  of percentage  frequency  distributions  of  direction  difference  for 
each  station,  average  direction  difference  for  each  station,  and  number  of  valid  counts  comprising  each  distribu¬ 
tion.  and  number  of  polls  of  data  indicating  a  FLAG  value  for  each  station. 


Speed -Ratio  (  Sensor  Speed  /  Mean  Speed) 

Figure  3-3.  Percentage  frequency  distribution  of  wind  speed  ratio  for  a  large  sample  of  data 
polls  from  the  LLWAS  anemometer  network  in  Orlando,  FL. 

These  profiles  were  used  empirically  to  establish  reasonable  parameter  values  against  which  the 
speed  and  direction  profiles  would  be  tested  for  deviations.  The  parameters  were  selected  conserva¬ 
tively  such  that  a  large  deviation  was  required  to  indicate  a  sensor  fault.  The  following  paragraphs 
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Figure  3—4.  Percentage  frequency  distribution  of  wind  direction  difference  from  mean  for  a  large 
sample  of  data  polls  from  the  LLWAS  anemometer  network  in  Orlando,  FL. 


describe  the  rationale  for  establishing  the  threshold  tests  to  monitor  speed  and  direction  data  quality 
using  these  parameters  and  the  accepted  FAA  guidelines. 

3.3.1  Speed  Tests 

Speed  tests  consist  of  examination  of  a  sensor’s  average  speed  ratio  for  wind  from  all  directions, 
its  distribution  of  speed  ratios,  and  its  variability  of  speed  ratio  value  with  mean  wind  direction.  Fig¬ 
ure  3-5  shows  the  conceptual  logic  employed  in  diagnosing  a  wind  speed  data  quality  problem.  The 
following  paragraphs  describe  the  specific  implementation  of  the  various  speed  threshold  tests. 


3.3.1. 1  Severe  or  catastrophic  malfunction 

Severe  malfunctions  that  are  either  electrical  or  mechanical  in  nature  often  result  in  extreme 
values  of  wind  speed,  often  manifested  as  reporting  of  a  constant  near-zero  value.  As  a  test  for  this 
type  of  problem,  wind  speed  distributions  are  examined  for  an  unexpectedly  high  proportion  of 
speed  ratios  occurring  in  the  extreme  low  and/or  speed  ratio  bins.  From  the  large  data  sample,  the 
expected  occurrence  of  wind  speed  ratios  less  than  0.50  or  greiter  than  1 .50  was  approximately  four 
percent  and  three  percent,  respectively.  For  our  catastrophic  speed  test  using  a  24-hour  period  of 
data  (which  would  be  expected  to  exhibit  a  larger  variance  due  to  its  smaller  sample  size),  a  speed 
sensing  problem  is  indicated  if  the  percentage  frequency  of  either  of  these  extreme  speed  bins  ex¬ 
ceeds  25  percent.  For  this  type  of  severe  error,  a  message  of  “extreme  low”  or  ‘‘extreme  high”  speed 
bias  is  generated. 

3. 3. 1.2  Speed  Bias  (frictional  drag,  improper  sensor  height,  sheltering) 

These  tests  examine  the  wind  statistics  for  a  low  or  high  bias  in  the  wind  speed  estimate.  An 
average  deviation  of  20  percent  or  more  (adaptable  parameter)  from  the  standard  wind  speed  is  con- 


14 


Speed  Sensing  OK 


Figure  3-5.  Conceptual  logic  applied  to  testing  for  and  diagnosing  speed  sensing  problems. 


sidered  an  unacceptable  bias.  Potential  causes  of  wind  speed  biases  include  undue  friction  in  the 
sensor  bearings,  a  sensor  that  is  sited  either  too  low  or  too  high,  or  physical  obstructions  that  shelter 
the  sensor. 

The  overall  average  speed  ratio  of  wind  from  all  directions  (0  to  360  degrees)  is  examined  for 
a  low  or  high  speed  bias.  If  it  is  less  than  a  0.80  (representing  a  20  percent  low  departure  from  stan¬ 
dard),  then  a  low  speed  bias  is  indicated.  If  it  is  greater  than  1 .20  (representing  a  20  percent  high 
departure),  then  a  high  speed  bias  is  indicated.  If  a  low  bias  indication  persists  for  several  days,  the 
speed  disuibutions  of  the  individual  direction  bins  are  then  examined  using  a  larger  data  sample 
(10-30  days)  to  determine  whether  the  speed  bias  shows  some  directional  dependence.  The  long 
duration  for  accumulating  data  for  this  test  is  required  so  that  sufficient  samples  are  available  for 
winds  from  all  directions.  The  speed  ratio  distribution  for  each  wind  direction  is  then  tested  sepa¬ 
rately,  and  a  message  is  reported  to  indicate  if  the  speed  bias  showed  any  dependency  on  wind  direc¬ 
tion.  If  the  unacceptably  low  bias  is  apparent  for  only  a  subset  of  all  wind  directions,  then  the  prob¬ 
lem  is  considered  to  be  directionally  dependent  and  atuibuted  to  sheltering  by  nearby  obstructions. 
Lf  the  low  bias  shows  no  directional  dependence,  the  problem  is  attributed  to  either  frictional  drag 
on  the  sensor  or  an  insufficient  pole  height. 

3.3.2  Direction  Tests 

Direction  tests  consist  of  examination  of  a  sensor’s  average  direction  difference  from  the  mean 
and  the  frequency  distribution  of  direction  differences.  Figure  3-6  shows  the  logic  employed  in 
diagnosing  a  direction  sensing  problem.  The  following  paragraphs  provide  and  explanation  of  the 
various  direction  threshold  tests. 
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Direction  Sensing  OK 


Figure  3-6.  Conceptual  logic  applied  to  testing  for  and  diagnosing  direction  sensing  problems. 


3. 3.2.1  Severe  or  Catastrophic  Malfunction 

As  with  wind  speed,  severe  malfunctions  that  are  either  electrical  or  mechanical  in  nature  often 
result  in  extreme  departure  in  wind  direction  values  from  the  mean  direction.  From  the  large  sample 
of  data,  extreme  direction  differences  (greater  than  95  degrees)  occurred  with  a  frequency  of  less 
than  one  percent.  For  the  daily  test  for  catastrophic  direction  problems,  an  error  is  indicated  it'  this 
extreme  condition  occurs  with  a  frequency  of  greater  than  five  percent. 

33.2.2  Directional  Offset  ( improper  orientation,  loose  mounting,  sticky  bearing) 

This  test  checks  for  an  off  set  in  the  wind  direction  sensor  from  the  true  wind  direction.  A  direc¬ 
tion  offset  is  usually  associated  with  a  direction  sensor  that  is  not  properly  onented  (i.e.,  incorrect 
ground  reference),  one  whose  mounting  is  loose,  or  with  a  "stickiness"  in  the  direction  bearings  that 
cause  a  jcrkincss  or  lag  in  movement  of  the  wind  direction  vane.  For  each  sensor,  the  average  direc¬ 
tion  difference  is  compared  against  a  parameter  indicating  the  maximum  allowable  difference.  Any 
persistent  daily  difference  greater  than  15  degrees  is  considered  unacceptable. 

33.2.3  Flat  Distribution  (loose  mounting,  sticky  bearing) 

If  a  sensor  is  experiencing  a  loose  mounting  or  a  sticky  bearing,  over  time  the  average  difference 
from  the  true  direction  may  not  stray  far  from  zero,  as  errors  of  different  sign  (i.e.,  clockwise  or  coun- 
teiclockwise)  may  tend  to  offset  one  another.  However,  the  problem  would  still  be  evident  as  a 
broadened  distribution  (higher  variance)  in  fne  wind  direction  frequency  profile.  Thus,  even  for  ac¬ 
ceptably  small  average  direction  differences,  the  difference  profile  was  tested  for  variance. 
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To  test  for  variance,  the  percentage  frequency  of  direction  difference  counts  of  less  than  1 5  de¬ 
grees  and  less  than  35  degrees  are  examined.  From  the  large  data  sample,  one  would  expect  that  the 
direction  difference  to  be  less  than  15  degrees  approximately  80  percent  of  the  time,  and  less  than 
35  degrees  approximately  90  percent  of  the  time.  For  the  threshold  test  on  these  standards,  a  distribu¬ 
tion  is  considered  to  have  an  unacceptably  broad  distribution  if  less  than  45  percent  of  the  counts 
have  a  direction  difference  of  plus/minus  15  degrees,  or  if  less  than  80  percent  of  the  counts  have 
a  direction  difference  of  plus/minus  35  degrees. 

3.4  Reporting  Sensor  Status 

Using  the  logic  and  threshold  tests  described  for  wind  speed  and  direction,  a  summary  page  that 
indicates  the  status  of  each  sensor  is  automatically  generated  for  each  day  of  data.  For  each  sensor, 
the  status  summary  page  indicates  the  number  of  valid  counts  for  compiling  speed  ratio  data,  the 
genera]  speed  sensing  status  (good,  bad,  or  unknown),  the  average  speed  ratio,  a  text  comment  re¬ 
garding  any  bad  speed  status  message,  the  direction  dependency  of  any  bad  speed  problem,  the  gen¬ 
eral  direction  sensing  status,  and  the  average  direction  difference.  A  sample  summary  page  is  shown 
in  Figure  3-7.  In  addition,  a  summary  of  the  speed  and  direction  profile  characteristics  for  each  sen¬ 
sor  is  also  generated  in  order  to  further  investigate  any  deficiency  indicated  on  the  summary  page. 
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Fig, iic  3-7.  Sample  summary  indicating  sensor  status. 
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4.  TESTING  THE  DQA  APPROACH 


During  1991-1993,  MIT  Lincoln  Laboratory  conducted  an  operational  demonstration  of  the 
Terminal  Doppler  Weather  Radar  (TDWR)  system  at  Orlando  International  Airport.  There  were  two 
anemometer  networks  deployed  in  association  with  these  demonstration  and  testing  efforts.  The  first 
was  the  1 4-station  Phase  III  Network  Expansion  LLWAS  system  owned  and  operated  by  the  FA  A. 
This  system  was  capable  of  operating  both  as  a  stand-alone  wind  shear  detection  system  for  MCO, 
and  as  a  component  of  the  TDWR/LLWAS  integration  system  that  was  being  demonstrated.  As  part 
of  the  Phase  III  LLWAS  upgrade  at  MCO,  the  anemometers  were  mounted  on  tall  towers  ranging 
from  1 20  to  1 50  feet  in  height  to  minimize  sheltering  effects  in  the  airport  terminal  area.  The  second 
anemometer  network  was  the  15-sensor  mesonet  (mcsoscale  network)  operated  by  Lincoln  Labora¬ 
tory.  Data  from  this  network  was  collected  off  line  in  support  of  the  TDWR/LLWAS  testing  effort. 
These  sensors  were  raised  on  portable  aluminum  towers  and  were  situated  approximately  90  feet 
above  ground  level  to  lessen  the  sheltering  effects  in  the  heavily-wooded  central  Florida  environ¬ 
ment.  An  illustr  ation  of  both  anemometer  networks  in  relationship  to  the  runways  at  MCO  is  shown 
in  Figure  4-1. 


Figure  4-1.  Location  of  FAA  and  MIT  Lincoln  Laboratory  anemometer  networks  during  1992  at  MCO. 
LI  through  LI 5  (bold  face  type)  indicate  locations  of  14  FAA  LLWAS  sensors  (Station  L9  not  sited). 

Ml  through  M15  indicate  locations  of  15  Lincoln  Laboratory  mesonet  sensors. 


Although  some  feasibility  testing  of  the  DQA  was  done  during  1991,  most  of  the  analysis  of 
the  automated  methodology  reported  here  was  performed  using  data  collected  during  1992  and 
1993.  During  1992,  anemometer  data  were  collected  from  mid-April  through  mid-September.  At 
the  end  of  each  day  (0000  UTC),  the  data  from  both  networks  were  processed  off  line  by  the  DQA 
software,  and  the  output  summaries  were  examined.  Periodically,  a  data  sample  comprising  several 
(usually  2-5)  days  of  data  was  processed  to  more  closely  examine  instances  of  marginal  sensor  per- 
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formance  or  to  better  determine  speed  sensing  dependency  on  wind  direction.  A  summary  log  of  the 
speed  and  direction  status  of  each  sensor  was  maintained  throughout  the  five-month  period.  When 
data  quality  problems  were  clearly  evident,  the  appropriate  maintenance  personnel  (FAA  or  Lincoln 
Laboratory,  depending  on  the  network)  were  notified.  At  the  end  of  the  data  collection  period,  sever¬ 
al  very  laige  samples  of  data  (ranging  from  10  to  30  days  each)  were  processed  to  further  examine 
sensor  performance  and  assess  the  effectiveness  of  the  DQA  method. 

Following  the  1992  data  collection  season,  the  Phase  III  configuration  of  LLWAS  was  down¬ 
graded  to  a  Phase  II  six-sensor  configuration,  while  the  Lincoln  Laboratory  mesonet  was  removed 
(for  deployment  at  another  site).  However,  during  1993  the  FAA  restored  off-line  data  collection 
of  the  1 4-sensor  Phase  III  LLWAS  configuration  as  part  of  a  system  performance  evaluation  for  the 
Phase  III  LLWAS  contractor.  DQA  processing  was  also  performed  on  much  of  this  data. 

The  following  paragraphs  discuss  observations  of  DQA  performance,  focusing  on  the  1992  and 
1993  datasets.  Thisdiscussion  includes  some  general  observations  as  well  as  examples  of  individual 
sensor  performance  in  detecting  both  severe  and  non-severc  data  quality  problems. 

4.1  General  Observations 

The  DQA  was  initially  nun  using  data  from  a  merged  file  of  synchronized  data  from  all  29 
anemometers,  i.e.,  using  LLWAS  and  Lincoln  mesonet  sensors  combined.  The  first  few  data  quality 
checks  in  April  of  1992  indicated  a  distinct  speed  sensing  difference  between  the  two  networks.  The 
automated  daily  status  summary  typically  indicated  nearly  half  of  the  sensors  as  having  a  speed  sens¬ 
ing  problem;  many  of  the  LLWAS  sensors  were  showing  unacceptably  high  speed  biases,  while  the 
Lincoln  sensors  had  low  biases.  Examining  several  days  of  data,  the  average  speed  ratio  of  the 
LLWAS  sensors  was  1 .07;  that  of  the  Lincoln  sensors  was  0.88.  It  was  concluded  that  this  character¬ 
istic  difference  of  approximately  15  percent  was  primarily  attributable  to  the  different  pole  heights 
of  the  two  networks.  The  LLWAS  anemometers,  located  120  to  150  feet  above  the  ground,  in  general 
were  safely  above  obstructions  in  the  terminal  area  (with  some  exceptions).  The  Lincoln  sensors, 
although  raised  90  feet  above  ground  level,  were  at  a  lower  effective  height  relative  to  the  tree  cano¬ 
pies  that  are  plentiful  in  central  Florida.  In  one  sense,  this  result  served  as  justification  for  the  FAA 
decision  to  raise  the  LLWAS  sensors  on  the  tall  poles  (incurring  additional  site  expense),  as  the 
LLWAS  sensors  were  viewed  as  providing  a  better  measurement  of  the  operational  winds.  To  avoid 
the  effect  of  the  differing  speed  sensing  characteristics  of  the  two  networks,  the  DQA  software  was 
run  separately  each  day  for  each  network. 

With  the  DQA  being  run  separately  for  LLWAS  and  Lincoln  sensors,  it  was  found  to  be  very 
successful  in  indicating  severe  sensor  malfunctions  such  as  grossly  underestimated  wind  speeds  or 
largely  erroneous  wind  directions.  There  were  several  instances  of  these  severe  failures,  particularly 
in  the  Lincoln  mesonet,  due  to  a  variety  of  problems  such  as  faulty  cables,  lew  power  supplies,  light¬ 
ning  strikes,  misoriented  ground  reference  for  direction  sensing,  etc.  These  extreme  problems  pri¬ 
marily  occurred  toward  the  beginning  of  the  data  collection  period  when  the  Lincoln  network  was 
being  stabilized.  More  subtle  data  quality  problems  were  a  greater  challenge,  as  expected,  but  the 
DQA  gave  reliable  indications  of  when  a  potential  problem  existed.  Most  of  the  difficulty  in  provid¬ 
ing  a  timely  indication  of  the  more  subtle  problems  arose  from  the  limitations  of  using  single-day 
data  samples.  The  two  primary  limitations  were:  1 )  many  days  included  a  small  sample  size  because 
of  light  wind  conditions  (i.e.,  mean  wind  speed  of  less  than  3  m/s)  not  uncommon  in  central  Florida, 
and  7)  even  for  days  with  large  samples,  the  mean  wind  direction  is  usually  limited  to  a  modest  range 
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(typically  30-60  degrees),  so  it  is  virtually  impossible  to  make  judgement  on  the  direction  dependen¬ 
cy  of  speed-sensing  performance.  In  spite  of  the  limitations  of  working  with  single-day  samples, 
analysis  of  large  samples  covering  several  days  were  quite  reliable  and  consistent  in  assessing  speed¬ 
sensing  performance. 

After  analyzing  both  large  and  small  samples  from  both  the  1 992  and  1 993  data  sets,  it  was  clear 
that  the  most  effective  implementation  of  the  DQA  would  be  to  vary  the  sample  size  and  thresholds 
for  various  tests.  It  is  clear  that  some  problems  are  evident  in  a  very  small  data  sample  (an  hour  or 
so),  some  require  a  very  large  sample  (several  days,  or  even  weeks),  while  others  require  a  sample 
size  somewhere  in  between.  In  addition,  some  tests  could  be  done  using  more  than  one  sample  size, 
applying  less  stringent  thresholds  to  the  smaller  sample.  Details  for  this  type  of  implementation  are 
given  in  Section  5.  These  recommendations  were  included  in  development  of  the  DQA  functional 
requirements  provided  to  the  FAA. 

4.2  Examples  of  Irregular  Sensor  Performance 

The  following  paragraphs  cite  some  specific  examples  from  the  1992  and  1993  data  sets  that 
illustrate  the  performance  of  the  automated  DQA. 

4.2.1  Catastrophic  Speed  Sensing  Failure 

As  mentioned,  there  were  several  instances  where  an  anemometer  experienced  a  severe  prob¬ 
lem,  usually  evidenced  by  extremely  low  wind  speed  values.  Oftentimes  these  problems  result  in 
reporting  of  a  continuous  zero  or  near-zero  wind  speed,  which  is  evident  in  a  very  small  data  sample. 
Problems  of  these  types  are  characterized  by  a  discrete  change  in  sensing  performance  rather  than 
a  gradual  degradation. 

An  example  of  a  catastrophic  speed  sensing  failure  identified  by  the  automated  DQA  occurred 
on  1  May  1992.  The  average  speed  ratio  for  that  day  was  reported  at  0.09  (see  Figure  4-2).  In  addi¬ 
tion  to  a  low  value  of  the  average  speed  ratio,  an  indication  of  the  critical  nature  of  the  failure  was 
provided  by  the  90  percent  frequency  value  in  the  “extreme  low”  speed  ratio  bin,  i.e.,  a  speed  ratio 
of  less  than  0.50  was  occurring  90  percent  of  the  time,  far  greater  than  the  25  percent  threshold.  The 
sensor  site  was  examined,  and  the  problem  was  isolated  to  a  faulty  cable  to  the  speed  sensor.  The 
cable  was  replaced  on  2  May;  the  output  statistics  from  3  May  (Figure  4-2)  showed  no  data  quality 
problems.  Extreme  speed  sensing  problems  of  this  nature  were  apparent  in  small  sample  sizes,  and 
the  DQA  methodology  was  very  effective  in  providing  a  timely  indication  of  failure. 

4.2.2  Speed  Biases  and  Direction  Dependency 

In  contrast  to  the  extreme  speed-sensing  failures,  there  were  several  instances  in  which  an 
anemometer  indicated  a  speed  ratio  that  was  just  outside  the  acceptable  0.80  to  1.20  range  for  only 
a  one-  or  two-day  period,  and  yet  was  satisfactory  most  of  the  time.  Once  again,  this  was  more  evi¬ 
dent  in  the  Lincoln  sensor  network.  From  these  single-day  indications,  it  was  unclear  whether  the 
overall  performance  of  the  sensor  should  be  considered  acceptable.  It  was  assumed  that  the  short¬ 
term  indication  of  inferior  performance  was  due  to  two  possible  effects:  1 )  the  problem  was  direc¬ 
tion-dependent  and  the  wind  was  blowing  from  a  more  sheltered  direction  on  those  days,  or  2)  the 
overall  performance  of  the  sensor  was  marginal  and  the  intermittent  failure  indications  were  a  conse- 
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Figure  4-2.  Speed  sensing  statistical  data  for  Station  M3,  for  1  May  (uui  3  May  1993.  Included  are  station 
number,  table  of  percentage  frequency  distributions  of  speed  ratio  sorted  by  mean  wind  direction  bin,  average 
speed  ratio  for  each  wind  diiection,  and  number  of  valid  counts  comprising  the  distribution  for  each  wind 
direction  bin. 


qucnce  of  the  day-to-day  variability  of  the  wind  field  over  the  network,  i.c.,  on  some  days  the  wind 
is  more  turbulent  or  gusty,  thus  causing  a  larger  standard  deviation  of  wind  values  across  the  network 
for  any  given  data  poll.  In  light  of  these  instances,  it  became  clear  that  future  versions  of  the  algo¬ 
rithm  should  include  threshold  testing  on  different  time-tiers  (i.c.,  short-  and  long-term  tests)  as 
will  be  described  in  Section  5.  The  next  few  paragraphs  show  examples  of  how  direction  dependen¬ 
cy  affects  the  testing  for  speed  biases. 
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4.2.2. 1  Example  of  Low  Speed  Bias  with  Direction  Dependence 

An  example  of  a  sensor  with  a  low  speed  bias  that  exhibited  direction  dependence  was  observed 
with  Station  M4  of  the  Lincoln  mesonet.  Figure  4-3  shows  a  plot  of  the  daily  speed  ratio  during 
12-21  June.  The  plot  also  shows  the  predominant  mean  wind  direction  for  each  day.  Over  this 
10-day  period,  the  average  speed  ratio  was  0.90,  safely  within  the  acceptable  20  percent  criterion. 
However,  on  16  June,  the  daily  speed  ratio  was  0.78,  which  is  considered  unacceptably  low.  Since 
this  was  , the  only  day  of  the  10- day  sample  during  which  the  wind  blew  from  120  degrees,  the  im¬ 
plication  is  that  the  sensor  experienced  an  unacceptably  high  amount  of  sheltering  from  that  direc¬ 
tion.  Although  this  is  apparent  in  the  10-day  sample,  the  sample  is  still  limited  in  the  range  of  direc¬ 
tions  covered;  furthermore,  the  sample  sizes  from  many  of  directions  that  are  covered  are  sparse. 
As  a  matter  of  fact,  there  were  only  61 1  valid  polls  of  data  from  the  120  degree  mean  direction  bin 
on  the  16th,  which  does  not  qualify  as  a  sufficient  sample  for  an  individual  direction.  (720  polls  is 
considered  a  sample  size  minimum;  there  were  1175  valid  polls  from  any  direction  on  that  day, 
which  yielded  the  0.78  daily  speed  ratio  value.  The  speed  ratio  using  only  the  611  polls  from  120 
degrees  was  0.71.) 


Figure  4-3.  Overall  daily  speed  ratio  for  Station  M4  during  12-21  June  1992.  Numbers  in  parentheses 
indicate  predominant  wind  direction  (in  degrees)  on  each  day. 


It  was  clear  that  reliable  conclusions  regarding  directional  dependency  of  speed  sensing  perfor¬ 
mance  require  a  much  larger  sample.  In  order  to  better  understand  this  dependency  and  its  variability 
from  station-to-station,  several  very  large  samples  of  data  were  DQA  processed  at  the  end  of  the 
data  collection  period.  The  results  of  these  large  sample  statistics  are  shown  in  Figures  4-4  and  4-5. 
The  graphs  in  Figure  4-4  were  compiled  using  data  from  1 3  of  the  Lincoln  mesonet  sensors  between 
the  period  21  May  to  17  June  1992  (the  other  two  Lincoln  anemometers  were  inoperable  during  this 
period),  representing  a  sample  of  more  than  25,000  valid  polls,  and  includes  a  sufficient  sample  of 
data  for  all  mean  wind  direction  bins.  The  figure  shows  the  speed  ratio  for  each  mean  direction  bin 
as  well  as  the  average  speed  ratio  for  all  directions  combined.  (This  overall  value  represents  a 
weighted  average  since  some  direction  bins  contain  more  samples  than  others.)  A  similar  represen¬ 
tation  for  the  14  FAA  LLWAS  ser.sor  is  shown  in  Figut  4-5.  This  sample  was  taken  during  6-22 
June  (except  for  Station  L5 ,  which  was  missing  much  data  during  this  particular  period)  and  includes 
approximately  40,000  polls  of  valid  data. 

These  distributions  give  a  better  perspective  of  single-day  indications  of  speed  sensing  defi¬ 
ciencies.  For  example,  if  wc  re-examine  Station  M4,  the  overall  speed  ratio  is  0.90  (same  as  that 
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Figure  4-4  Graphs  showing  long-term  average  speed  ratios  for  15  Lincoln  Laboratory  ntesonet 
anemometer  stations.  Each  graph  shows  the  speed  ratio  corresponding  to  each  of  the  1 2  mean 
wind  direction  bitu<,  plus  the  overall  speed  ratio  for  ah  directions. 
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Figure  4-4  (Continued). 
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Figure  4-4  (Continued). 
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Figure  4-5.  Graphs  showing  long-term  average  speed  ratios  for  14  FAA  LLWAS  anemometer  sta¬ 
tions.  Each  graph  shows  the  speed  ratio  corresponding  to  each  of  the  12  mean  wind  direction  bins, 
plies  the  overall  speed  ratio  for  all  directions. 
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Figure  4-5  (Continued). 
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of  the  smaller  10-day  sample),  with  the  lowest  speed  ratio  of  0.80  occurring  for  the  120degree  mean 
wind  direction  bln.  This  is  consistent  with  our  observation  of  the  low  speed  ratio  of  0.78  on  the  16th, 
when  the  predominant  wind  was  from  120  degrees.  The  next  question,  however,  is  whether  this  sen¬ 
sor’s  performance  should  be  considered  acceptable  (for  the  moment  disregarding  the  fact  that  the 
entire  Lincoln  mesonet  is  considered  to  have  a  low  speed  problem  due  to  insufficient  tower  height). 
The  larger  sample  indicates  that  the  performance  is  acceptable,  albeit  marginal  from  a  couple  of 
directions  (0.80  speed  ratio  from  120 degrees,  0.81  from  330),  while  the  single-day  sample  implies 
a  sensor  deficiency.  For  implementation  as  an  on-line  monitoring  system,  it  was  concluded  that  the 
best  approach  would  be  to  apply  a  less  stringent  threshold  to  the  single-day  sample  and  apply  the 
stricter  threshold  to  a  larger  sample.  Thus,  a  deviation  from  the  mean  would  be  required  to  be  more 
“convincing”  in  the  one-day  sample  in  order  to  justify  action  to  be  taken  by  maintenance  personnel. 

4.22.2  An  example  of  direction  dependence  speed  bias  for  an  LLWAS  sensor 

Only  two  of  the  FAA  LLWAS  sensors  indicated  potential  speed  sensor  problems  worthy  of  note. 
The  first  involved  Station  L10,  which  is  of  special  significance  as  it  represented  the  Centerfield  sen¬ 
sor  of  the  network,  which  is  the  wind  value  that  is  provided  routinely  to  pilots.  It  also  illustrates  the 
importance  of  directional  dependence  of  speed  performance  monitoring  by  the  DQA.  For  the  first 
part  of  the  summer.  Station  L10  showed  no  indication  of  a  potential  sensor  problem.  This  is  consis¬ 
tent  with  the  large  sample  distribution  shown  in  Fig  4-5i.  The  average  speed  ratio  overall  directions 
was  0.94;  the  value  of  speed  ratio  over  the  twelve  different  direction  bins  ranged  from  0.86  for  a 
southerly  wind  (180  degrees)  to  1.05  for  a  southeasterly  wind.  There  is  also  a  fair  amount  of  shelter¬ 
ing  from  directions  ranging  from  north-northwest  clockwise  around  to  cast,  with  speed  ratios  near 
0.90.  A  channelling  effect  is  apparent  for  winds  from  the  southeast,  with  ratios  greater  than  1 .00  for 
winds  in  the  120  and  150  degree  direction  bins.  This  distribution  with  direction  is  consistent  with 
the  local  obstructions  in  the  area  near  Sensor  L10.  There  are  airport  terminal  buildings  to  the  south 
of  the  anemometer  and  some  sheltering  by  trees  to  the  west,  north,  and  cast.  The  buildings  seem 
responsible  for  the  sheltering  from  the  south,  with  a  potential  channeling  effect  for  winds  from  the 
southwest  and  southeast. 

During  the  latter  portion  of  the  data  collection  period,  there  were  frequent  days  for  which  the 
DQA  indicated  a  low  speed  bias  for  Station  L10,  with  daily  speed  ratios  reaching  as  low  as  0.72. 
A  second  large  sample  of  over  80,000  polls  was  processed  using  data  collected  from  17  August  to 
15  September.  Figure  4-6  shows  distribution  of  speed  ratios  over  the  twelve  direction  bins  for  this 
period  as  compared  to  that  of  the  previous  large  sample  (6-22  June).  This  second  sample  shows  the 
overall  speed  ratio  to  be  0.85.  The  directional  dependence  of  the  speeds  shows  tendencies  similar 
to  that  of  the  first  sample,  but  more  exaggerated  in  amplitude.  For  instance,  the  sheltering  from  the 
north  around  to  the  east  appears  much  more  severe,  with  speed  ratios  ranging  from  0.75  to  0.82. 
In  addition,  the  channeling  from  the  southeast  and  southwest  is  also  exaggerated,  with  values  as  high 
as  1.10.  Although  these  exaggerations  are  not  clearly  understood,  the  significant  reduction  in  overall 
speed  ratios  (i.e.,  from  all  directions)  from  0.94  to  0.85  is  primarily  due  to  the  persistent  northeast 
wind  that  predominated  during  the  latter  data  collection  sample,  thus  weighting  the  overall  average 
with  winds  from  the  more  sheltered  directions.  If  we  compare  the  speed  ratios  for  the  two  samples 
using  equal  weighting  to  each  wind  direction  (irrespective  of  the  number  of  polls  from  each  direc¬ 
tion),  the  speed  ratio  difference  of  the  two  samples  is  insignificant  (0.93  versus  0.92).  This  does  not, 
however,  account  for  the  difference  in  speed  sensing  performance  within  individual  direction  bins 
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Figure  4—6.  Average  speed  ratio  corresponding  io  each  mean  wind  direction  bin,  for  two  separate 
samples  of  data. 

using  the  two  samples,  e.g.,  why  was  the  sheltering  from  the  northeast  more  severe  in  the  later  sam 
pic?  (It  is  possible  that  in  addition  to  sheltering,  the  sensor  also  experienced  a  gradual  degradation 
from  some  other  source,  such  as  a  speed  bearing  problem.)  Due  to  the  limited  availability  of  land 
in  the  central  portion  of  the  runway  area,  there  are  virtually  no  options  in  finding  a  superior  site,  and 
obstruction  height  restrictions  in  the  immediate  runway  area  preclude  raising  the  sensor  further,  so 
no  further  corrective  action  was  taken  during  1992. 

4.22.3  An  example  showing  deficiency  in  the  direction  dependence  test 

The  other  LLWaS  sensor  exhibiting  questionable  performance  during  1992  was  Station  L14. 
The  DQA  algorithm  identified  this  sensor  as  having  a  direction  dependent  speed  bias.  Although  it 
wascorrect  in  identifying  a  speed  bias,  closet  examination  indicated  that  the  bias  was  not  directional¬ 
ly  dependent.  This  example  has  led  to  proposal  of  an  improved  method  of  determining  direction  de¬ 
pendence,  as  described  in  the  following  paragraphs. 

Referring  back  to  Figure  4-5m,  the  overall  long-term  speed  ratio  from  all  directions  for  Station 
L14  is  0.82,  which  is  marginally  acceptable.  The  figure  shows  that  the  sensor  performance  would 
be  considered  acceptable  from  some  directions  (greater  than  0.80  ratio)  but  not  for  others  (less  than 
0.80  ratio),  and  thus  the  low  speed  bias  would  be  judged  by  the  DQA  algorithm  as  demonstrating 
a  directional  dependency.  However,  the  range  of  speed  ratios  ever  the  spectrum  of  direction  bins 
is  much  smaller  than  that  seen  in  the  previous  example  (Station  L10).  This  flatter  distribution  with 
direction  would  imply  that  there  is  little  direction  dependency  and  that  the  reason  a  subset  of  direc¬ 
tions  appear  acceptable  is  because  the  overall  speed  ratio  (0.82)  is  so  close  to  the  acceptable  thresh¬ 
old.  From  this  observation,  it  became  apparent  that  the  determination  of  directional  dependency 
should  be  based  on  tne  range  of  speed  ratio  values  over  all  of  the  direction  bins.  Toeslablish  a  typical 
range  of  values  with  direction,  we  used  the  large  samples  and  computed  for  each  sensor  the  differ¬ 
ence  between  the.  overall  speed  ratio  (from  all  directions  combined)  and  the  speed  ratio  of  the  direc¬ 
tion  bin  that  differed  farthest  from  the  overall  average  ratio.  The  distribution  of  these  difference 
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values  for  the  Lincoln  and  FAA  sensors  is  shown  in  Figure  4-7.  For  the  Lincoln  mesonet,  Stations 
M 1 ,  M4,  and  M 10  show  the  least  directional  dependence,  as  the  difference  between  the  overall  speed 
ratio  and  the  “worst"  direction  ratio  is  0. 1 0.  Station  M8  shows  the  most  directional  dependence,  with 
a  difference  of  0.20.  The  median  difference  value  for  the  Lincoln  mesonet  was  0.14.  Directional 
dependence  of  speed  performance  for  the  FAA  LLWAS  network  was  less  evident,  with  a  median 
difference  value  of  0.095.  Only  station  LI  exhibited  extreme  directional  dependence,  with  a  differ¬ 
ence  value  of  0.22.  Referring  back  to  its  speed  ratio  distribution  (Figure  4-5a),  Station  LI  showed 
an  overall  high  speed  bias  of  13  percent;  its  most  sheltered  direction  was  from  030  degrees,  for 
which  its  speed  ratio  was  1 .09.  In  contrast,  the  speed  ratio  from  330  degrees  was  a  dramatic  1 .38, 
the  laigest  deviation  from  average  for  any  station  oi  direction  for  either  network.  It  is  presumed  that 
this  is  an  effect  of  wind  channeling  around  the  terminal  building  located  to  the  north.  (This  is  the 
same  building  that  is  responsible  for  the  wind  channeling  effect  on  Station  LI  0  for  a  wind  with  south¬ 
erly  component.)  In  any  event,  it  seems  appropriate  that  these  difference  values  should  be  used  to 
determine  directional  dependence.  For  instance,  the  standard  deviation  of  difference  values  over 
the  LLWAS  network  is  0.038,  so  an  empirically-derived  threshold  for  a  direction-dependence  test 
might  be  the  mean  difference  (0.095  in  this  example)  plus  one  standard  deviation.  In  this  example, 
this  approach  would  yield  a  threshold  of  approximately  0.13,  so  Stations  LI,  L5,  and  L6  would  be 
considered  to  have  a  speed-sensing  performance  that  varies  significantly  with  direction. 

4.22.4  The  importance  of  inter-stalion  dependency  in  assessing  directional  dependence 

One  final  issue  regarding  directional  dependence  of  speed  sensing  has  to  do  with  inter-station 
dependency.  Looking  back  a  Station  LI  and  its  large  high  bias  from  the  330  degree  direction  bin, 
it  is  clear  that  an  effect  of  this  station  would  be  to  raise  the  network  mean  wind  speed,  thus  giving 
the  appearance  of  increased  sheltering  at  all  other  stations.  In  order  for  the  DQA  to  be  effective  in 
this  regard,  the  network  must  be  sufficiently  large  so  that  the  contribution  from  a  single  station  does 
not  overwhelm  the  network  mean  speed  value  and  create  artificial  speed  ratios  at  the  remaining  sen¬ 
sors.  In  this  example.  Station  LI  has  a  speed  ratio  value  for  330  degrees  that  is  0.22  greater  than 
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figure  4—  7  Maximum  speed  ratio  difference  defined  for  each  sensor  as  the  maximum  difference  in 
speed  ratio  between  the  overall  average  speed  ratio  (all  directions)  ami  the  average  speed  ratio  corre¬ 
sponding  to  each  of  the  12  mean  wind  direction  bins 
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its  overall  average.  If  this  were  the  only  non-meteorological  effect  contr  ibuting  to  an  increase  in 
the  network  mean  speed,  its  impact  would  be  an  artificial  increase  in  speed  ratio  of  nearly  0.02  foi 
the  other  stations  for  winds  from  330  degrees.  If  this  were  a  six-sensor  network,  the  impact  would 
be  a  speed  ratio  decrease  of  0.04  for  the  remaining  five  sensots.  Although  not  catastrophic  to  the 
methodology,  it  lends  support  to  an  approach  of  varying  the  suingency  of  the  speed  ratio  thresholds 
to  account  for  this  inter-station  dependency.  Perhaps  less  strict  speed  ratio  thresholds  should  be 
employed  by  smaller  networks  (like  the  six-sensor  Phase  II  system)  than  those  used  for  the  larger 
(12-20  sensor)  Phase  III  networks.  The  tradeoff  would  be  a  decrease  in  false  indications  of  a  bad 
sensor,  at  the  expense  of  a  lower  (or  less  timely)  detection  rate  of  faulty  sensors.  (Of  course,  for  the 
largest  networks,  e.g.,  greater  than  20  sensors,  the  larger  areal  extent  introduces  additional  variance 
in  wind  speeds  within  the  network  due  to  greater  horizontal  displacement,  as  described  earlier.)  We 
must  also  consider  that,  in  contrast  to  the  simple  example  involving  Station  LI ,  the  combined  effects 
of  local  sheltering  and  obstructions  of  all  the  sensors  within  a  network  is  far  more  complex.  Figure 
4-8  shows  a  scatter  diagram  of  wind  speed  ratios  of  all  1 4  LLWAS  anemometers  for  each  of  the  12 
direction  binsand  provides  some  sense  of  directional  dependence  and  the  sensitivity  of  inter-station 
dependence  of  speed  ratio  values.  Based  on  the  distribution  of  speed  ratio  values,  the  speed  sensing 
performance  of  the  network  as  a  whole  can  be  seen  to  more  “well-behaved"  for  some  wind  directions 
compared  to  others.  This  behavior  has  an  impact  on  the  effectiveness  of  the  DQA  methodology. 
For  example,  for  the  030  degree  direction  bin,  the  0.80  speed  ratio  (Station  L3)  is  a  clear  outl  icr  from 
the  rest  of  the  neiw  oik,  while  the  0.77  value  from  330dcgrees  (Station  LI  3)  is  at  least  partly  exagger- 
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ated  by  the  extreme  high  speed  bias  ( 1 .38  ratio)  of  Station  LI .  This  example  lends  additional  support 
for  choosing  less  stringent  thresholds,  particularly  for  the  small  sample  tests. 

4.23  Examples  of  Direction-Independent  Non-catastrophic  Speed  Failures 

Two  excellent  examples  of  detection  of  degraded  speed-sensing  performance  that  v>ere  inde¬ 
pendent  of  wind  direction  were  encountered  in  the  1993  data  sample.  The  first  is  a  straightforward 
example  of  low  speed  readings  from  Station  L7  (see  Figure  4-9).  The  speed  at  this  station  was  con¬ 
sistently  half  that  of  the  rest  of  network,  with  no  apparent  dependence  on  wind  direction.  FA  A  per¬ 
sonnel  investigated  the  sensor  and  determined  that  it  had  been  struck  by  lightning,  verifying  sub¬ 
standard  performance.  The  other  example  during  1993  involved  Station  L2.  Speed  sensing 
performance  was  considered  marginal  during  early  June,  with  speed  values  approximately  80  per¬ 
cent  of  the  network  mean  value  (Figure  4-10).  A  sample  taken  later  in  the  summer  indicated  that 
performance  had  degraded  significantly,  with  typical  speed  ratio  value  of  approximately  0.65.  Once 
again,  FAA  personnel  investigated  arid  verified  the  problem,  isolating  the  cause  as  excessive  friction 
in  the  speed  sensor  bearings,  a  condition  that  was  slowly  deteriorating  with  time. 
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Figure  4-9.  Speed  ratios  for  Station  L7.  Values  represent  a  composite  of  several  samples  taken  during  1993. 
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Figure  4-10.  Speed  ratios  for  Station  12,  showing  values  corresponding  to  m  o  separate  samples 
taken  during  1993. 
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4.2.4  Direction  Sensing  Failures 

There  were  several  instances  of  individual  sensors  showing  evidence  of  some  problems  in  sens¬ 
ing  wind  direction.  Once  again,  most  of  the  problems  were  with  the  Lincoln  niesonet,  while  the  FAA 
LLWAS  sensors  were  superior  in  performance.  Unfortunately,  many  of  the  direction-sensing  prob¬ 
lems  were  not  diagnosed  and  were  simply  correcred  by  equipment  lcplacemcnt.  One  good  example 
involving  station  M2,  however,  was  diagnosed  in  early  June.  The  daily  direction  displacement  from 
the  mean  for  Station  M2  ranged  from  10  to  60  degrees.  A  plot  of  the  percentage  frequency  distribu¬ 
tion  versus  direction  difference  bin  for  Station  M2  on  2  June  1992  is  shown  in  Figure  4-11.  The 
figure  also  shows  the  corresponding  distribution  for  all  stations  in  the  network  combined.  On  this 
day,  the  average  direction  difference  from  mean  was  27  degrees,  well  above  the  acceptable  threshold 
of  15  degrees  for  a  one-day  sample.  Equally  important  is  the  flatter  distribution  compared  to  that 
of  the  entire  network  combined.  This  sensor  was  found  to  have  a  loose  direction  mounting  that  was 
allowing  slippage  in  the  wind  vane  as  it  turned.  The  significance  of  the  statistical  distribution  is  note¬ 
worthy  for  this  type  of  failure  since  it  is  conceivable  that  a  wind  vane  could  exhibit  slippage  that 
results  in  a  near-zero  average  direction  difference  from  mean,  as  errors  of  different  sign  tend  to  can¬ 
cel  one  another.  This  same  effect  may  also  be  evident  for  a  sensor  with  a  sticky  direction  bearing 
that  causes  a  lag  in  response  to  wind  direction  changes.  These  examples  identify  the  need  to  examine 
the  distribution  of  direction  differences  in  addition  to  the  average  value.  The  implementation  of  a 
new  variance  test  for  this  purpose  is  discussed  in  the  next  section. 


□  Station  M2 

2  June  1992  fll  Stations  Ml -Ml  5 


Figu.C  4-11.  Percentage  frequency  distribution  o)  wind  direction  differences  for  Station  M2  compared  to 
distribution  using  all  sensors  within  the  mesonci. 
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5.  IMPROVEMENT  TO  DQA  DESIGN 


5*1  General 

The  automated  DQA  was  successful  during  1992  in  providing  a  timely  (day  by  day)  assessment 
of  anemometer  data  quality  and  demonstrated  feasibility  for  automated  on-line  performance  moni¬ 
toring.  When  used  in  conjunction  with  human  review  to  eliminate  overwaming,  the  method  resulted 
in  many  identifications  of  faulty  sensors,  with  no  clear  evidence  of  a  false  alert  (though  some  of  the 
alerts  of  faulty  sensors  or  inadequate  data  quality  were  inconclusive).  However,  improvement  to 
the  DQA  design  is  necessary  to  eliminate  the  step  of  manual  intervention  in  eliminating  false  alens 
of  a  sensor  failure.  This  section  presents  several  proposed  improvements  leading  toward  greatei  ef¬ 
fectiveness  both  as  an  off-line  analysis  tool  and  for  on-line  performance  monitoring.  These  recom¬ 
mended  improvements  are  provided  as  guidelines  for  further  development  and  refinement  of  the 
DQA  methodology  to  be  performed  at  the  FA  A  Technical  Center. 

Experience  indicates  the  greatest  potential  improvement  to  the  DQA  methodology  would  be  to 
change  the  implementation  of  sampling  frequency  (and  sample  sizes)  and  test  thresholds.  The  DQA 
algorithm  was  run  once  per  day  during  1992  testing,  using  a  single  set  of  thresholds.  Since  sonic 
malfunctions  become  apparent  in  a  data  sample  smaller  than  one  day  and  others  require  samples 
much  larger  than  a  single  day,  the  accumulated  statistical  data  base  should  be  varied  according  to 
test  type.  In  addition,  some  tests  should  be  performed  for  more  than  one  size  data  sample,  using 
thresholds  that  vary  with  sample  size.  Selecting  less  suingent  thresholds  for  smaller  samples  and 
more  stringent  thresholds  for  laiger  samples  would  help  to  eliminate  most  of  the  potential  false 
alarming  due  to  expected  fluctuations  resulting  from  the  daily  variation  in  the  meteorological  envi¬ 
ronment.  The  following  paragraphs  discuss  suggested  alterations  to  the  DQA  methodology.  The 
improved  implementation  presented  here  was  used  as  the  baseline  in  developing  the  system  func¬ 
tional  requirements  delivered  to  the  FAA. 

5.2  Accumulating  Data  Samples  of  Varying  Size 

The  accumulation  of  statistical  data  for  wind  speed  ratios  and  wind  direction  differences  should 
be  partitioned  into  small,  medium,  and  large  samples.  Regulation  of  data  accumulation  and  testing 
frequency  should  be  keyed  to  a  running  count  of  “valid”  data  polls,  i.c.,  data  polls  for  which  the  net¬ 
work  mean  wind  speed  is  above  the  minimum  speed  threshold.  (Keep  in  mind  that  many  one-day 
samples  of  data  consisted  of  vm y  few  “valid”  data  polls  due  to  light  winds).  This  tunning  count  is 
then  used  to  regulate  the  accumulation  of  data  in  shot!-,  medium-,  and  long-term  counters.  Counts 
for  speed  ratio  and  direction  difference  are  initially  recorded  in  short-term  counters;  after  a  suffi¬ 
cient  number  of  polls  have  accumulated,  the  counts  from  the  short-term  counters  arc  added  to  the 
medium-term  counter.  Short-term  threshold  tests  at  e  then  performed,  after  which  the  short-term 
counters  arc  re-initialized.  The  same  approach  is  then  used  for  passing  medium-term  counters  into 
the  long  term  counters.  The  overall  count  of  valid  polls  (polls  for  which  the  standard  wind  speed 
is  above  the  minimum)  thus  regulates  passing  of  data  from  one  set  of  counters  to  the  next,  as  well 
as  frequency  of  performance  of  short-,  medium,  and  long-term  threshold  tests. 

It  is  envisioned  that  smali-samplc  tests  wili  require  about  a  half-hour’s  worth  of  valid  polls 
(-200),  the  medium  sample  tests  will  require  several  hours  of  valid  polls  (-2000),  and  the  large  sain- 
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pie  tests  will  require  at  least  several  days  of  valid  polls  (-50,000).  Since  only  a  fraction  of  the  total 
polls  are  of  sufficient  speed  to  be  considered  valid,  the  actual  time  to  compile  this  number  of  valid 
polls  may  be  considerably  longer,  e.g.,  the  accumulation  of  50,000  valid  polls  may  require  weeks 
of  data.  In  order  to  improve  timeliness  for  warnings,  a  sliding  window  for  data  analysis  and  testing 
should  also  be  considered. 


5.3  Sample  Size  and  Thresholds  for  Specific  Malfunction  Tests 

This  section  describes  suggested  data  sample  sizes  and  thresholds  to  be  used  for  each  of  the  vari¬ 
ous  wind  speed  and  direction  tests.  The  thresholds  cited  here  are  estimates  and  may  need  to  be  re¬ 
fined  (cither  empirically  or  theoretically,  or  both)  to  ensure  proper  statistical  significance. 

Figure  5-1  is  a  summary  of  the  testing  logic  and  thresholds  that  represent  an  improvement  of 
the  DQA  design  using  multiple  sample  sizes  and  test  thresholds.  This  logic  is  described  in  the  follow¬ 
ing  paragraphs. 

5.3.1  Severe  Wind  Speed  Sensor  Malfunction 

This  test  should  be  done  frequently,  using  the  small  data  sample.  The  average  speed  ratio  is 
examined  for  a  very  large  (>50  percent)  deviation  from  unity.  The  result  is  a  frequently  performed 
test  (several  times  per  day)  for  a  severe  failure. 


SMALL-SAMPLE  TESTS  <200  VALID  DATA  POLLS) 

IF  SPD_RATI0  <  0.50.  — >  Severe  electro-mechanical  failure  of  speed  sensor 
IF  |DIP_0IFF|  >  30  DEGREES.  — >  Severe  electro-mechanical  failure  of  direction  sensor 


te£QlUfA£AMPLE-TESTS  (2P0P  POLLS) 

IF  SPD_RATIO  <  0.70.  — >  Frictional  drag  or  sensor  too  low 
IF  SPD_RATIO>  1.30,  — >  Sensor  height  too  high 

IF  DIR_0lFF  >20  deg  AND  VARIANCE  IS  LOW  — >  Direction  sensoi  misoriented 
IF  DIR_DIFF  >  20  deg  AND  VARIANCE  IS  HIGH  — >  Loose  mounting  or  sticky  direction  bearing 
IF  DIR_DIFF  <20  deg  AND  VARIANCE  IS  HIGH  — >  Loose  mounting  or  sticky  direction  bearing 


LARGE  SAMPLE-&Q4QQJKflLLS) 

IF  jSPD_RATIO(overall)  -  SPD  RATIO(worst  direction)!  >  0.15  — >  Directional  dependence 
IF  SPD_RATlO(any  direction)  <“0.80  AND  DIRECTION  DEPENDENCE  — >  Sheltering 
IF  SPD  RATIO  <  0.80  AND  NO  DIRECTION  DEPENDENCE  — >  Frictional  drag  or  sensor  too  low 
IF  SPD  RATIC'overall)  >  1.20  AND  DIRECTION  DEPENDENCE  — >  Wind  channeling 
IF  SPD_RATIO>  1  2.1  AND  NO  DIRECTION  DEPENDENCE  — >  Sensor  too  high 
IF  DIR_DIFF  >  10  dv.q  AND  VARIANCE  IS  HIGH  — >  Direction  sensor  misoriented 
IF  DIR..DIFF  >  10  ueg  AND  VARIANCE  IS  LOW  — >  Improper  electrical  grounding 
IF  DIR_DIFF  <  to  deg  AND  VARIANCE  IS  HIGH  — >  Loose  mounting  or  sticky  direction  bearing 


Figure  5-1.  Summary  of  logic  associated  with  use  of  multiple  sample  sizes  and  test  thresholds  for 
malfunctioning  testing,  as  proposed  for  improved  DQA  design.  Actual  values  of  sample  sizes  and 
thresholds  aw  estimates. 
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5.3;2  Speed  Bias  from  Frictional  Drag  or  Improper  Sensor  Height 

FAA  Technical  Order  6560.21  A  establishes  the  criteria  that  the  sheltering  at  a  sensor  shall  not 
exceed  20  percent.  Based  on  this  criteria,  wc  have  adopted  the  rule  that  the  average  speed  ratio  for 
a  sensor  shall  not  be  less  than  0.80  or  exceed  1 .20.  This  standard,  however,  assumes  a  stable  estimate 
of  speed  ratio  for  a  station.  Due  to  expected  day-to-day  variability  in  wind  speed  ratio  estimation, 
sensors  with  an  acceptable  stable  (long-term)  value  of  speed  ratio  may  experience  some  days  with 
an  marginally  unacceptable  average  value  of  speed  ratio.  This  was  observed  many  times  during  1 992 
testing,  particularly  with  the  less  reliable  Lincoln  mesonet  sensors. 

To  avoid  false  alerting  of  unacceptable  speed  bias  based  on  a  single  day  sample  of  data,  the  best 
approach  appears  to  be  to  test  for  speed  bias  using  both  the  medium  (several  hours)  and  large  (several 
days)  samples.  For  the  medium-size  sample,  the  approach  is  to  apply  a  less  stringent  standard  (say 
a  30  percent  deviation  from  the  standard  speed)  so  that  a  failed  test  would  imply  with  reasonably 
high  confidence  that  a  more  stable  estimate  (larger  sample)  would,  in  fact,  yield  a  value  tnat  is  not 
within  the  +/-  20  percent  acceptable  range.  This  same  test  would  also  be  done  using  the  large  sample 
data,  applying  the  more  stringent  20  percent  threshold.  The  benefit  of  testing  on  the  medium-sized 
sample  in  addition  to  the  large  sample  is  that  it  allows  for  a  more  timely  indication  of  some  sensor 
malfunctions,  while  still  providing  some  safeguarding  against  frequent  false  alarming. 

5.33  Sheltering  from  Local  Obstruction 

The  test  for  directionally  dependent  sheltering  requires  a  large  data  sample,  so  that  a  reasonable 
sample  of  data  is  available  for  winds  from  a  wide  range  of  wind  directions.  During  1 992  testing,  a 
10-day  sample  of  data  was  analyzed  periodically  for  this  purpose.  Wind  speed  ratio  distributions 
were  tested  separately  for  each  wind  direction  bin.  The  logic  applied  was  that  if  the  average  wind 
speed  ratio  was  less  than  0.80  (more  than  20  percent  low)  for  only  a  subset  of  directions,  the  wind 
speed  problem  was  interpreted  as  being  directionally  dependent. 

Although  this  approach  of  testing  wind  speed  characteristics  separately  for  different  wind  direc¬ 
tions  had  some  merit,  a  refinement  to  the  logic  is  necessary.  For  instance,  if  a  sensor  was  showing 
speeds  that  were  2  J  percent  low  from  one  direction  and  1 9  percent  low  from  all  other  directions,  that 
would  be  interpreted  as  a  sheltering  problem.  A  better  approach  would  be  to  compare  the  speed  ratio 
for  each  direction  to  the  overall  speed  ratio.  Using  this  logic,  a  station  would  be  considered  sheltered 
if  it  was  at  least  20  percent  low  for  a  subset  of  directions  and  its  “worst”  direction  was  significantly 
lower  than  the  average  speed  ratio  from  all  directions  combined.  In  the  previous  example,  the  data 
would  not  suggest  sheltering,  since  the  unacceptable  wind  speed  from  the  single  direction  wasn’t 
much  worse  than  the  “acceptable”  speeds  from  the  other  directions.  A  directionally  dependent  speed 
bias  of  15  percent  seems  a  reasonable  value  for  this  purpose,  as  supported  by  the  empirical  example 
described  in  4.2.2.  In  other  words,  sheltering  would  be  indicated  if  a  station  exhibited  at  least 
20  percent  low  speed  bias  for  only  a  subset  of  directions,  and  the  speed  ratio  corresponding  to  the 
worst  direction  was  at  least  15  percent  lower  than  the  overall  (i.c.,  from  all  directions)  speed  ratio. 

5.3.4  Severe  Direction  Sensor  Malfunction 

As  with  the  test  for  severe  speed  sensor  malfunction,  this  test  would  be  done  frequently,  using 
the  small  data  sample.  The  average  direction  difference  would  be  examined  for  large  (>30  degrees) 
deviation  from  unity. 
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5.3.5  Directional  Offset  and  High  Variance  of  Direction  Difference  Values  (Improper 
Orientation,  Loose  Mounting,  High  Frictional  Drag) 

As  with  the  tests  for  speed  bias,  tests  foi  directional  offset  should  be  done  on  both  the  medium 
and  large  data  samples,  varying  the  threshold  with  sample  size.  A  long-term  (i.e„  using  the  large 
data  sample)  stable  estimate  of  more  than  10  degrees  wind  direction  difference  from  the  standard 
wind  direction  would  be  consioered  unacceptable.  A  less  stringent  threshold  of  20  degrees  differ¬ 
ence  would  be  used  for  the  medium-sized  sample. 

In  addition  to  testing  for  directional  offset,  a  test  cf  the  distribution  of  direction  difference  val¬ 
ues  is  also  necessary.  This  was  done  during  1992  by  examining  the  cumulative  frequencies  within 
certain  direction  difference  bins.  In  retrospect,  it  seems  more  appropriate  to  compute  a  mathematical 
variance  of  direction  difference  values  and  develop  an  acceptable  variance  threshold.  Directional 
offset  with  an  acceptable  variance  would  imply  an  orientation  problem,  while  an  offset  with  an  unac¬ 
ceptably  high  variance  would  imply  a  loose  mounting  or  sticky  bearings.  Even  without  directional 
offset,  these  two  latter  problems  could  potentially  exist  with  no  directional  offset  if  the  variance  was 
unacceptably  high. 
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6.  CONCLUSIONS  AND  RECOMMENDATIONS 


During  1991,  manual  analysis  of  wind  statistics  demonstrated  the  feasibility  of  using  character¬ 
istic  departures  from  network  mean  speed  and  direction  to  identify  and  isolate  potential  LLWAS  sen¬ 
sor  malfunctions.  A  series  of  logical  tests  was  developed  to  automate  the  statistical  analysis  proce¬ 
dure.  Automated  data  quality  analysis  software  was  run  Offline  each  day  for  several  months  during 
1992  using  data  from  both  the  1 4-station  enhanced  LLWAS  network  and  the  15-station  Lincoln  me- 
sonet  at  Orlando,  FL;  additional  LLWAS  data  was  examined  during  1993.  It  was  found  that  the  anal¬ 
ysis  automation  was  effective  in  timely  identification  of  severe  sensor  malfunctions.  The  method 
also  showed  effectiveness  in  identifying  more  subtle  data  quality  deficiencies  when  a  sufficiently 
large  data  sample  was  available  for  analysis.  However,  assessment  of  sensor  performance  from  a 
single  day’s  data  was  more  difficult  when  data  quality  appeared  marginal,  and  further  manual  analy¬ 
sis  of  the  statistical  wind  data  over  a  larger  sampling  period  was  necessary  to  avoid  false  indications 
of  a  sensor  problem.  As  a  result,  it  became  evident  that  implementation  of  an  on-line  system  would 
best  be  applied  using  a  more  conservative  alerting  approach  in  order  to  minimize  unnecessary  re¬ 
sponse  of  maintenance  personnel  for  false  alerts.  Consistent  with  this  philosophy  and  based  on  expe¬ 
rience  gained  during  testing,  improvements  to  the  DQA  method  have  been  suggested.  The  most  fun¬ 
damental  improvement  is  to  implement  the  method  using  test  thresholds  that  vary  with  sample  size. 
The  variation  in  sample  size  allows  timeliness  for  detection  of  problems  that  are  evident  in  a  small 
sample  of  data  as  well  as  providing  sufficiently  large  samples  for  identifying  more  subtle  problems. 
Variation  of  test  thresholds  with  sample  size  is  a  reflection  of  the  different  level  of  confidence  af¬ 
forded  by  different  sample  sizes,  which  is  of  particular  importance  when  the  quality  of  data  appears 
marginal. 

The  recommendation  presented  here  is  that  the  FAA  begin  by  developing  an  off-line  automated 
DQA  methodology  as  outlined  in  this  report  and  perform  regular  data  quality  analysis  at  a  central 
facility  for  anemometers  from  remote  network  sites.  Once  acceptable  thresholds  are  determined  for 
individual  sites  and  the  DQA  approach  is  refined,  this  methodology  should  be  implemented  as  an 
on-line  performance  monitoring  system,  with  sensor  failure  messages  reported  directly  to  the 
LLWAS  Remote  Maintenance  Monitoring  System  display  so  that  more  timely  corrective  action  may 
be  taken  by  on-site  personnel. 
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APPENDIX 


Automated  LLWAS  Data  Quality  Analysis 
Functional  Requirements  Document 

19  February  1993 


1.  GENERAL 

1.1  Scope 

This  document  establishes  the  functional  requirements  for  development  of  an  automated  Data 
Quality  Analysis  (DQA)  method  for  detection  of  malfunctioning  Low  Level  Windshear  Alert  Sys¬ 
tem  (LLWAS)  sensors.  The  DQA  is  intended  to  operate  in  real  time  in  conjunction  with  the  LLWAS 
Wind  Shear/  Microburst  (WSMB)  detection  algorithms. 

1.2  Introduction 

The  Phase  II  LLWAS  is  currently  the  primary  mechanism  for  detection  of  hazardous  wind  shear 
in  the  terminal  area  at  more  than  100  airports  throughout  the  United  States.  In  addition  to  wind  shear 
information,  it  also  provide"  surface  observations  for  the  airport  centerfield  wind  and  ninway 
threshold  winds.  It  employs  a  network  of  six  anemometers  surrounding  the  airport  terminal  area. 
Over  the  next  several  years,  a  more  sophisticated  and  reliable  system  (Phase  III)  will  be  installed 
at  more  than  half  of  the  existing  sites.  The  improved  system  will  typically  require  12  to  24  sensors 
per  site.  By  the  end  of  the  decade  there  will  be  well  over  1 000  LLWAS  senors  at  airports  throughout 
the  country. 

The  quality  of  wind  data  is  important  for  the  proper  functioning  of  LLWAS.  Filters  in  the 
LLWAS  algorithms  provide  protection  against  occasional  spurious  wind  measurements.  However, 
systematic  bias  in  the  wind  measurements,  which  is  difficult  to  detect  by  manual  data  inspection, 
can  cause  degraded  wind  shear  detection  performance  either  by  causing  missed  detections  or  by 
causing  false  alerts.  In  addition,  LLWAS  sensor  networks  are  designed  to  provide  minimal  redun¬ 
dant  coverage  for  wind  shear  detection.  Thus,  high  data  quality  from  each  sensor  in  the  network 
is  required  for  proper  system  performance.  In  the  absence  of  wind  shear,  however,  there  is  redundan¬ 
cy  in  the  wind  shear  measurement  by  the  sensors  within  the  network;  this  redundancy  can  be  used 
as  a  basis  for  identification  of  sensors  that  are  not  functioning  properly.  This  document  describes 
the  requirements  of  an  automated  method  for  detection  of  degraded  sensor  performance  based  on 
the  statistical  comparison  of  the  data  from  each  sensor  with  that  from  the  full  network  when  wind 
shear  is  not  present  over  the  network. 

1.3  Reference  Documents 

This  document  references  FAA  document  ‘'Network  Expansion  (Phase  III)  Algorithm  Specifi¬ 
cation,  Version  1990.02.”  The  LLWAS  Algorithm  Specification  supercedes  any  conflict  with  this 
functional  requirements  document. 
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2.  DQA  DESCRIPTION 
2.1  Relationship  with  LLWAS 

The  DQA  is  intended  to  operate  on-line  in  parallel  with  the  LLWAS  Phase  III  Network  Expan¬ 
sion  algorithm.  As  such,  it  uses  the  LLWAS  data  stream  for  wind  data  acquisition  and  may  access 
LLWAS  algorithm  parameters  from  the  Airport  Configuration  File  (ACF)  as  well  as  input/output 
variable  values  from  LLWAS  function  modules  (Figure  2-1). 


Figure  A2-1.  Relationship  between  processing  of  wind  data  by  LLWAS  Algorithm  ( top  row)  and  auto¬ 
mated  sensor  Data  Qualiry  Analysis  (bottom  row). 


2.2  Algor  thm  Description 

The  basic  approach  of  the  DQA  is  to  detect  a  systematic  departure  in  the  wind  data  from  a  single 
sensor  with  that  from  some  standard  wind,  where  the  standard  wind  is  defined  as  the  mean  wind 
derived  from  all  of  the  sensors  within  the  local  network  For  each  poll  of  LLWAS  data,  the  ratio  of 
wind  speed  at  each  sensor  and  the  standard  wind  speed  is  computed.  The  difference  between  each 
sensor’s  wind  direction  and  the  standard  direction  is  also  computed.  Using  these  speed-ratios  and 
direction-differences,  a  statistical  data  base  is  compiled  for  each  station.  For  wind  speed-ratio,  sta¬ 
tistics  arc  maintained  separately  for  various  ranges  of  standard  wind  direction,  since  speed  sensing 
problems  may  be  directionally  dependent,  e.g„  due  to  sheltering  from  a  nearby  obstruction.  For  each 
station,  the  statistical  information  includes  the  mean  and  sample  standard  deviation  of  speed-ratio 
and  direction  difference  values. 

Depending  on  sensor  design,  the  basic  sensor  measurements  may  include  one  or  more  wind 
speeds  (e.g.,  direct  measurement  of  vector  speed  versus  separate  measurements  of  speed  compo¬ 
nents).  Speed  statistics  must  be  compiled  separately  for  each  type  of  speed  measurement. 

Using  the  statistical  information,  threshold  tests  are  performed  periodically  to  identify  a  signifi¬ 
cant  departure  from  the  standard  wind.  The  frequency  of  testing  depends  on  test  type.  In  all,  there 
are  three  levels  of  testing,  whereby  each  level  is  characterized  by  the  frequency  with  which  the  test 
is  performed  and  the  size  of  the  data  sample  required  for  the  test.  For  example,  tests  to  identify  severe 
differences  from  the  standard  wind  speed  or  direction  are  performed  relatively  frequently,  using  a 
small  sample  of  data  and  lenient  test  thresholds.  Tests  to  identify  more  subtle  departures  are  per¬ 
formed  using  a  large  sample  of  data  and  more  stringent  thresholds.  Data  sample  sizes  will  range  from 


less  than  an  hour  for  the  small  sample,  several  hours  for  the  medium  sample,  and  more  than  a  week 
for  the  large  sample.  Table  2-1  indicates  the  characteristics  of  the  wind  data  departures  from  the 
standard  that  arc  indicative  of  a  sensor  problem  and  the  size  of  the  corresponding  data  sample  re¬ 
quired  to  perform  a  legitimate  test. 


TABLE  A2*1 

Wind  Data  Characteristics  indicative  of  a  Sensor  Malfunction 


Wind  Data  Characteristic 

Sample  required 

Extreme  high  or  low  wind  speeds 

small 

Speed  bias 

medium,  large 

Directional  dependence  of  speed  bias 

large 

Extreme  direction  offset 

small 

Extremely  high  variance  of  direction-difference  values 

smail 

Direction  ofrset 

medium,  large 

High  variance  of  direction-difference  values 

medium,  large 

Determination  of  wind  data  characteristics  from  the  small-,  medium-,  and  large-sample  tests 
can  then  be  used  individually  and  in  conjunction  with  one  another  to  isolate  the  characteristics  of 
a  sensor  malfunction. 

3.  FUNCTIONAL  REQUIREMENTS 

This  section  defines  the  system  functional  requirements  necessary  for  proper  implementation 
of  the  DQA  algorithm. 

3.1  General 

3.1.1  Parameters 

The  DQA  software  requires  a  number  of  adjustable  parameters.  DQA  adjustable  parameters 
shall  be  included  within  the  LLWAS  Airport  Configuration  File  (ACF).  The  DQA  shall  be  able  to 
access  all  parameters  from  the  local  LLWAS  ACF.  A  description  of  adaptable  parameters  required 
by  the  DQA  is  shown  in  Table  3-1. 

3.1.2  LLWAS  I/O  Variables 

The  DQA  shall  be  able  to  access  all  LLWAS  I/O  variable  values,  as  listed  for  each  module  with¬ 
in  the  Network  Expansion  (Phase  HI)  Algorithm  Specification. 

3.1.3  Algorithm  Initialization 

The  DQA  shall  be  initialized  upon  initialization  or  re-initialization  of  the  LLWAS  WSMB  al¬ 
gorithm. 
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CO  CO 


TA3LE  A3-1 

Description  of  Adaptable  Parameters  Required  by  the  DQA  Algorithm. 
Each  parameter  is  Described  by  a  Paragraph  Reference  to  This 
Document,  its  Units,  the  Value  Type  (fpd=floating  point  decimal, 
int=integer),  its  Default  Value,  and  its  Range  of  Possible  Values. 


Para 

Parameter  Description 

Units 

Type 

Default 

Range  of  Values 

direction  bin  parameters 

3.2.3 

Nurnty'-  of  standaid  direction  bins 

none 

int 

36 

1  to  72 

3.2.3 

Direction  bin  initial  angle 

deg 

tpd 

iG.O’(n-l) 
(n=1  to  36] 

0.0  to  360.0 

3.2.3 

Direction  bin  final  angle 

deg 

fpd 

10.0‘n 
(n=1  to  36] 

0.0  to  360.0 

VALID  POLL  THRESHOLDS 

3.2.4 

Minimum  speed  threshold  for  valid  poll 

m/s 

fpd 

3.0 

0.0  to  10.0 

PARAMETERS  FOR  TEST  FREQUENCY  AND  SAMPLE  SIZE 

3.4.1 

Period  elapsed  between  short-term  tests 

polls 

int 

200 

100  to  1000 

3.3.1 

Small  sample  size 

polls 

int 

200 

100  to  1000 

3.4.1 

Period  elapsed  between  medium-term  tests 

polls 

int 

2000 

100  to  10,000 

3.3.1 

Medium  sample  size 

polls 

int 

2000 

500  to  10,000 

3.4.1 

Period  elapsed  between  long-term  tests 

polls 

int 

50.000 

10.000  to  100,000 

3.3.1 

Large  sample  size 

polls 

int 

50,000 

10,000  tol  00,000 

THRESHOLDS  FOR  SHORT-TERM  TESTS 
Sufficient  sensor  sample 

polls 

int 

150 

100  to  1000 

3.4.2.1 

Low  spaed-ratio  threshold 

none 

fpd 

0.50 

0.00  to  9.99 

3.4 .2.1 

High  speed-ratio  threshold 

none 

fpd 

2.00 

0.00  to  9.99 

3.4. 2.2 

Low  direction-difference  threshold 

deg 

fpd 

—45 

-99  toO 

3.4 .2.2 

High  direction-difference  threshold 

deg 

fpd 

+45 

0  to— 99 

THRESHOLDS  FOR  MEDIUM-TERM  TESTS 
Sufficient  sensor  sample 

polls 

int 

1000 

500  to  10,000 

3.4.2.1 

Low  speed-ratio  threshold 

none 

fpd 

0.75 

0.00  to  9.99 

3.4 .2.1 

High  speed-ratio  threshold 

none 

fpd 

1.25 

0.00  to  9 .99 

3.4 .2.2 

Low  direction-difference  threshold 

deg 

fpd 

-30.0 

-99.C  to  0.0 

3.4. 2.2 

High  direction-difference  threshold 

deg 

fpd 

+30.0 

0.0  to  -99.0 

3.4 .2.3 

Direction— dift  standard  deviation  threshold 

deg 

fpd 

15.0 

0.0  to  99.0 

THRESHOLDS  FOR  LONG-TERM  TESTS 
Sufficient  sensor  sample  (per  direction  bin) 

polls 

int 

1000 

1000  to  10.000 

3.4.2.1 

Low  speed  threshold 

none 

fpd 

0.80 

0.00  to  9.99 

3.4.2.1 

High  speed  threshold 

none 

fpd 

1.20 

0.00  to  9.99 

3.4. 2.2 

Low  direction  threshold 

deg 

fpd 

-15.0 

-99.0  to  0.0 

3.4 .2.2 

High  direction  threshold 

deg 

fpd 

+  15.0 

0.0  to  -99.0 

3.4 .2.3 

Direction— diff  standard  deviation  threshold 

deg 

fpd 

15.0 

0.0  to  99.0 

3.4.24 

Speed-ratio  difference  from  mean  (direction 

none 

fpo 

0.10 

0.0  to  9.9 

dependency  test) 
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3.1.4  Timing  Requirements 

All  DQA  processing  shall  be  performed  within  TBD  seconds  following  receipt  of  each  poll  of 
input  data. 

3.2  Data  Preparation /Preprocessing 

3.2.1  Input  Winds 

The  DQA  shall  use  as  input  the  raw  wind  values  acquired  in  real-time  by  LLWAS  prior  to  any 
data  preprocessing  within  the  LLWAS  algorithm.  The  design  of  the  DQA  will  have  some  dependen¬ 
cy  on  the  basic  sensor  wind  measurement,  which  is  in  turn  dependent  upon  sensor  design.  As  such, 
all  speed  data  processing  (speed  ratio  computations,  statistical  information  processing,  threshold 
testing)  referenced  throughout  this  document  shall  be  performed  on  all  basic  sensor  wind  speed 
measurements.  If  the  basic  sensor  measurement  does  not  include  a  direct  measurement  of  wind 
speed,  then  all  speed  data  processing  shall  also  be  performed  on  the  values  of  composite  speed  as 
derived  from  speed  component  measurements. 

3.2.2  Determination  of  Standard  Wind  Speed  and  Direction 

For  each  poll  of  LLWAS  data,  the  DQA  shall  determine  a  standard  wind  speed  and  direction 
The  DQA  shall  derive  the  standard  wind  as  the  mean  wind  speed  and  direction  of  all  sensors  within 
the  local  network,  using  the  mean  wind  component  values  (U_bar,  V _bar)  from  the  LLWAS  NET- 
WORK_MEAN  function. 

3.2.3  Standard  Direction  Bins 

Since  wind  speed  characteristics  may  be  directionally  dependent,  the  statistical  information  for 
wind  speed  must  be  derived  separately  for  winds  from  various  ranges  of  mean  wind  direction.  Stan¬ 
dard  direction  bins  are  defined  as  ranges  of  standard  winddirection.  The  range  of  directions  compris¬ 
ing  each  bin  shall  be  adaptable,  as  described  in  Table  3-1. 

3.2.4  Valid  Polls 

A  valid  poll  for  DQA  processing  purposes  is  defined  as  a  poll  of  LLWAS  data  for  which  the 
standard  wind  speed  is  at  least  some  adaptable  speed  threshold.  The  DQA  shall  monitor  a  count  of 
valid  polls  of  data  received  following  DQA  algorithm  initialization  or  re-initialization.  The  count 
of  valid  polls  shall  be  used  to  determine  the  frequency  of  performance  of  various  threshold  tests. 

3.2.5  Computing  Speed  Ratios 

For  each  valid  poll  of  data,  for  each  sensor  with  valid  (not  missing  or  flagged)  data,  the  DQA 
shall  compute  the  ratio  of  the  sensor  wind  speed  and  the  standard  speed. 

3.2.6  Computing  Direction  Differences 

For  each  valid  poll  of  data,  for  each  sensor  with  valid  (not  missing  or  flagged)  data  whose  wind 
speed  is  also  above  the  adaptable  speed  threshold,  the  DQA  shall  compute  the  difference  in  degrees 
between  the  sensor  wind  direction  and  the  standaid  wind  direction. 
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3.2.7  Suspending  Processing  During  Wind  Shear 

The  DQ  A  shall  suspend  processing  ot  wind  data  (statistical  data  accumulation  and  processing) 
while  evidence  of  wind  shear  exists  in  the  airport  vicinity.  Evidence  of  wind  shear  is  defined  as 
occurrence  of  either  of  the  following: 

a.  Existence  of  a  non-NULL  LLWAS  alert  message  for  any  runway  for  the  cur¬ 
rent  data  poll,  or 

b.  Value  greater  than  1.00  for  any  of  the  following  output  variables  from  the 
LLWAS 

DIVERGENCE_RATIO  function,  for  the  current  poll: 

Edge_dvrg_ratio(edge)  [edge-  1,NUM_EDGE) 
Edge_cvrg_ratio(edge)  [edge-  1.  NUM_EDGE] 
Tri..dvrg_ratio(m)  [tri- 1,NUM_TRI] 

Tri_cvrg_ratio(?ri)  [tri-  1,  NUM_TRI) 


3.3  Statistical  Information 

Statistical  information  describing  the  speed-ratio  and  direction-difference  values  computed  for 
each  sensor  shall  be  automatically  generated  on  a  periodic  basis.  This  statistical  information  shall 
be  used  for  threshold  testing  at  three  different  test  levels,  as  described  in  3.4.1. 

3.3.J  Data  Collection 

The  DQA  shall  collect  samples  of  wind  data  for  each  sensor  according  to  three  classes.  The 
three  classes  shall  be  categorized  based  on  adaptable  parameters  which  define  sample  size  and  sam¬ 
ple  update  frequency,  as  described  in  Table  3-1. 

3.3.2  Statistical  Information  Describing  Speed-Ratio 

The  statistical  information  describing  the  speed-ratio  values  for  each  sensor  shall  include  the 
following  lor  each  of  the  threes  sample  sizes: 

1 .  Mean  speed-ratio  values.  The  mean  speed-ratio  shall  be  computed  for  both: 

a.  each  standard  wind  direction  bin,  i.e.,  a  mean  speed-ratio  representative 
of  the  speed-ratio  values  that  were  computed  when  the  standard  wind  was 
blowing  from  the  range  of  directions  defined  by  each  bin,  and 

b.  speed  -ratio  values  computed  for  winds  from  any  direction,  i.c.,  all  direc¬ 
tion  bins  combined. 

2.  Number  of  polls  for  which  a  speed-ratio  value  was  computed,  foi  each  stan¬ 
dard  wind  direction  bin,  and  for  all  direction  bins  combined. 

3.  Sample  standard  deviation  of  speed-ratio  values,  computed  separately  for 
each  standard  wind  direction  bin,  and  for  all  speed-ratio  values  combined  re¬ 
gardless  of  standard  wind  direction. 
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3.33  Statistical  Information  Describing  Direction-Difference 

The  statistical  information  describing  the  direction-difference  values  for  each  sensor  shall  in¬ 
clude  the  following  for  each  of  the  three  sample  sizes: 

a.  Mean  direction-difference  value. 

b.  Number  of  polls  for  which  a  direction-difference  value  was  computed. 

c.  Sample  standard  deviation  of  direction-difference  values. 

3.4  Performance  of  Threshold  Tests 
3.4.1  Test  Levels 

Threshold  testing  shall  occur  on  three  levels:  short-,  medium-,  and  long-term  testing,  where 
each  of  the  three  test  levels  is  defined  by  the  frequency  of  performance  of  testing  (defined  in  terms 
of  elapsed  valid  polls)  and  the  size  of  the  data  sample  used  for  testing  (measured  in  valid  polls). 
Frequency  of  performance  and  data  sample  size  shall  be  adaptable  parameters,  as  described  in  Table 
3-1.  The  type  of  tests  performed  at  each  test  level  shall  be  as  listed  in  Table  3-2. 

TABLE  A 3*2 

Types  of  Tests  Performed  at  Each  Test  Level 

SHORTTERM  MEDIUM  TERM  LONGTERM 

SPEED  BIAS  XXX 

DIRECTION  OFFSET  XXX 

DIRECTION-DIFF  VARIANCE  X  X 

DIRECTION  DEPENDENCE  X 


3.4.2  Test  Types 

This  section  describes  tin*,  threshold  test  types  to  be  performed.  Performance  of  each  test  shall 
provide  an  indication  of  the  test  result  as  summarized  in  Table  3-3.  Indications  resulting  from  spe¬ 
cific  tests  are  described  in  tfo;  following  paragraphs. 

3. 4. 2.1  Speed  bias 

Testing  for  wind  speed  bias  shall  be  done  for  all  three  test  levels.  Fortach  stauon  with  sufficient 
polls  of  valid  data,  the  mean  speed-ratio  value  for  winds  from  any  direction  shall  be  compared  with 
a  low  threshold  value  and  a  high  threshold  value.  The  number  of  sufficient  valid  polls  and  the  thresh¬ 
old  valuer,  for  each  test  level  shall  be  adaptable  parameters  as  described  in  Table  3-1.  If  the  mean 
speed-ratio  value  is  less  than  the  low  threshold,  an  indication  of  low  speed  shall  be  given.  If  the  mean 
spced_ratio  value  is  greater  than  the  high  threshold,  an  indication  of  high  speed  shall  be  given. 
Otherwise,  an  indication  of  good  speed  shall  be  given.  For  the  long-term  test  level  using  the  large 
data  sample,  the  mc.an-spccd  ratio  for  each  of  the  standard  direction  bins  with  sufficient  valid  polls 
shall  also  be  compared  against  the  low  and  high  threshold  values.  A  separate  speed  indication,  as 
described  earlier  in  this  paragraph,  shall  be  given  for  each  direction  bin. 
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TABLE  A3-3 

Possible  Indications  Resulting  from  Small-, 
Medium-,  and  Large-Sample  Threshold  Yests 


POSSIBLE  INDICATIONS 


LOW,  HIGH.  GOOD 
LOW,  HIGH,  GOOD 


LOW,  HIGH,  GOOD 
LOW.  HIGH,  GOOD 
HIGH,  GOOD 


LOW.  HIGH.  GOOD 
LOW.  HIGH.  GOOD 
HIGH.  GOOD 
YES,  NO 


1.4.22  Direction  offset 

Testing  for  wind  direction  offset  shall  be  done  for  all  three  test  levels.  For  each  station  with 
sufficient  polls  of  valid  data,  the  mean  direction-difference  value  shall  be  compared  with  a  low 
threshold  value  and  a  high  threshold  value.  The  number  of  sufficient  valid  polls  and  the  threshold 
values  for  each  test  level  shall  be  adaptable  parameters  as  described  in  Table  3-1 .  If  the  mean  direc¬ 
tion-difference  value  is  less  than  the  low  threshold,  an  indication  of  low  direction  shall  be  given. 
If  the  mean  direction-difference  v.due  is  greater  than  the  high  threshold,  an  indication  high  direction 
shall  be  given.  Otherwise,  an  indication  of  good  direction  shall  be  given. 


TEST TYPE 

SMALL  SAMPLE  TESTS 

SPEED-RATIO 

DIRECTION-DIFFERENCE 


SPEED-RATIO 

DIRECTION-DIFFERENCE 

DIRECTION-DIFFERENCE  STANDARD  DEVIATION 

1ABSE.SA&SELE 

SPEED-RATIO 

DIRECTION-DIFFERENCE 

DIRECTION-DIFFERENCE  STANDARD  DEVIATION 
DIRECTION-DEPENDENCE  OF  SPEED 


3.42.3  Large  direction-difference  standard  deviation 

Testing  for  an  unacceptably  high  sample  standard  deviation  of  direction-difference  values  shall 
be  done  for  the  medium-  and  long-term  test  levels.  For  each  station  with  sufficient  polls  of  valid 
data,  the  sample  standard  deviation  of  direction-difference  values  shall  be  compared  to  a  threshold 
value.  The  number  of  sufficient  valid  polls  and  the  threshold  value  for  each  test  level  shall  be  adapt¬ 
able  parameters  as  described  in  Tabic  3-1 .  If  the  sample  standard  deviation  is  greater  than  the  thresh¬ 
old,  an  indication  of  high  direction  standard  deviation  shall  be  given.  Otherwise,  an  indication  of 
good  direction  standard  deviation  shall  be  given. 
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3.4.2  4  Directional  dependence  of  speed  sensing  performance 


Testing  for  directional  dependence  of  speed  sensing  performance  shall  be  done  for  the  long¬ 
term  test  level.  The  difference  between  the  mean  speed-ratio  value  corresponding  to  each  of  the 
standard  direction  bins  with  sufficient  valid  polls  and  the  mean  speed-ratio  value  for  winds  from 
airy  direction  shall  be  computed.  The  number  of  sufficient  polls  per  direction  bin  and  the  difference 
threshold  shall  be  adaptable  parameters  as  described  in  Table  3-1 .  An  indication  of  direction  depen¬ 
dence  shall  be  given  if  the  absolute  value  of  any  of  these  differences  is  greater  than  an  adaptable 
threshold.  Otherwise,  an  indication  of  no  direction  dependence  shall  be  given. 

3,5  Er«-or  Messages 

The  DQa  shall  use  the  indications  from  the  threshold  tests  to  provide  an  error  message  to  the 
LLWAS  maintenance  function.  The  logic  for  error  message  generation  shall  be  as  shown  in  Tabie 
3-4.  The  ASCII  text  associated  with  each  message  shall  be  adaptable. 
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TABLE  A3-4 

Logic  Applied  to  Test  Indications  for  Generation  of  Error  Messages 


TEST  INDICATIONS 


ERROR  MESSAGE 


1.  Spd-Ratio  -  LOW 

2.  Spd-Ratio  -  HIGH 

3.  Dir-Difference  -  LOW 

4.  Dir-Difference  -  HIGH 


Low  speed.  Replace  sensor 
High  speed.  Replace  sensor. 

Check  sensor  orientation.  Correct  or  replace  sensor. 
Check  sensor  orientation  Correct  or  replace  sensor. 


1.  Spd-Ratio  -  LOW.  and 
Dir-Depend  (from  large-sample  test)  - 

2.  Spd-Ratio  -  LOW,  and 
Oir-Depend  (from  large-sample  test)  ■ 

3.  Spd-Ratio  «=  HIGH,  and 
Dir-Depend(  from  large-sample  test) 

4.  Spd-Ratio  *  HIGH,  and 
Dir-Depend  (from  large-sample  test) 

5.  Dir-Difference  »  LOW,  and 

Dir— Diff  Standard  Deviation  =  GOOD 

6.  Dir-Difference  -  HIGH,  and 
Dir-Diff  Standard  Deviation  =  GOOD 

7.  Dir-Difference  *>  LOW,  and 
Dir-Diff  Standard  Deviation  =  HIGH 

8.  Dir-Difference  -  HIGH,  and 
Dir-Diff  Standard  Deviation  =  HIGH 


NO  Low  speed.  Replace  sensor. 

YES  Sheltering.  Notify  FAA  authority. 

NO  Improper  sensor  height.  Notify  FAA  authority. 

YES  Improper  sensor  height.  Notify  FAA  authority. 

Check  sensor  orientation.  Correct  or  replace  sensor. 
Check  sensor  orientation.  Correct  or  replace  sensor. 
Check  sensor  orientation.  Correct  or  replace  sensor. 
Check  sensor  orientation.  Correct  or  replace  sensor. 


1.  Spd-Ratio  ■=  LOW.  and 
Direction  Dependence  *  NO 

2.  Spd-Ratio  -  LOW,  and 
Direction  Dependence  <=  YES 

3.  Spd-Ratio  -  HIGH,  and 
Direction  Dependence  =  NO 

4.  Spd-Ratio  =  HIGH,  and 
Direction  Dependence  -  YES 

5.  Dir-Difference  -  LOW.  and 
Dir-Diff  Standard  Deviation  -  GOOD 

6.  Dir-Difference  «  HIGH,  and 
Dir-Diff  Standard  Deviation  GOOD 

7.  Dir-Difference  »  LOW,  and 
Dir-Diff  Standard  Deviation  «=  HIGH 

8.  Dir-Difference  -  HIGH,  and 

Dir— Dirt  Standard  Deviation  =  HIGH 


Low  speed.  Replace  sensor. 

Shelteiing.  Notify  FAA  authority. 

Impio,.  .  jnsor  height.  Notify  FAA  authority, 
improper  sensor  height.  Notify  FAA  authority. 

Check  sensor  orientation.  Correct  or  replace  sensor. 
Check  sensor  orientation.  Correct  or  replace  sensor. 
Check  sensor  orientation.  Correct  or  replace  sensor. 
Check  sensor  orientation.  Correct  or  replace  sensor. 


